Springer Nature has a platform for Research Communities, so after I published my paper Building a Modern Data Platform Based on the Data Lakehouse Architecture and Cloud-Native Ecosystem, I posted a blog post about the paper titled:
The blog post summaries the paper highlights with some personal background.
Enjoy :-)
Finally, after months of hard work, I have published my first research paper in a double-blind peer-reviewed scientific journal by the international publisher Springer Nature 🙌
The paper is titled:
Building a Modern Data Platform Based on the Data Lakehouse Architecture and Cloud-Native Ecosystem
This research paper is the result of several months of work and is based on my master's thesis, which was published in 2023 (I got Master of Science with Distinction in Data Engineering from Edinburgh Napier University).
The paper presents a practical application for data management without vendor lock-in, in addition to ensuring platform extensibility and incorporating modern concepts such as Cloud-Native, Cloud-Agnostic, and DataOps.
Why is this paper important? Because data is the backbone of Artificial Intelligence! In today's world, control over data means political and economic independence.
I would like to extend my sincere gratitude to the research team who contributed to this work, supported me, and shared their knowledge to help bring this paper to the highest quality. It was a truly enriching experience on many levels! 🙌
The research group chose these quotes from our respective languages/cultures to emphasize the importance of perseverance and diligence:
“عِندَ الصَّباحِ يَحمَدُ القومُ السُّرَى”
(In the morning, the people praise the night's journey)
Arabic Proverb
“Αρχή ήμισυ παντός”
(The beginning is half of
everything)
Greek Proverb
“Is obair latha tòiseachadh”
(Beginning is a
day's work)
Scottish Gaelic Proverb
I will write a community blog post about it soon :-)
Almost 3 years ago (September 2020), I enrolled in a part-time master's program in data engineering at Edinburgh Napier University. Finally, two weeks ago, I got an email informing me that I successfully completed my master's and the program board of examiners awarded me Master of Science with Distinction in Data Engineering 🎉🎉🎉
My graduation ceremony would have been today, but I postponed it to October 2023 for some personal matters. So it's just a matter of time till the official graduation.
So today, I'd like to share my master's dissertation: Modern Data Platform with DataOps, Kubernetes, and Cloud-Native Ecosystem.
The dissertation builds a proof of concept for the core of Modern Data Platform using DataOps, Kubernetes, and Cloud-Native ecosystem to build a resilient Big Data platform based on Data Lakehouse architecture, which is the basis for Machine Learning (MLOps) and Artificial Intelligence (AIOps).
It was a super challenging topic, given the fact that Data Lakehouse architecture emerged just in 2020! But it was great to dive into it. I've been into data for years, and it's exciting to step up my skills from T-Shaped to Pi-Shaped (DevOps and DataOps) :-)
About 10 years ago, I used SaltStack extensively and wrote my top 10 best practices for SaltStack formulas . Today, I do the ...
