Platform for Scalable Web Apps | How I built my website with Kubernetes

Platform for Scalable Web Apps | How I built my website with Kubernetes



I’ve spent months building a platform to support my development of scalable web applications. In this video, I try to share the learning process.

My personal website has a lot behind the scenes that most people cannot interact with. I’ve written up a few microservices and decided to deploy them on kubernetes. Some may say that what I’ve done is overkill – but it really does simplify the workflow once its setup (and I don’t have to rewrite my codebase in the future!)

The end result:
– Kubernetes cluster on digital ocean.. 3 nodes, a load balancer, and persistent volume
– nginx-ingress controller exposes services to the outside world (all other services are internal)

Microservices:
– jwt authentication server (security)
– hangfire background processing server
– MySQL database
– mailer
– front end

There is still a lot of work to be done. While most services are scalable – the database is not. Currently, our database is a single instance. I think someday I will need to deploy a MySQL cluster to support replication, etc. This was a massive topic to cover in a single video so I could not explain everything, my hope is that people can take what they see and research further.

Technologies Used:
– CPU Intensive Backend: C# with ASP .NET Core
– Simple I/O Intensive Backend: Node js
– Database: MySQL
– Scheduler: Hangfire open source background processing library
– Authentication: JWT backed by Dotnet Identity
– Mailing Service: Sendgrid
– Integrations: YouTube, Github
– Deployment: Docker images w/ Kubernetes
– Front end: Next js / React

Hangfire (background processing):
YouTube Descriptions Updater:
Next Video:
Previous Video:

Social
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Website:
Twitter:
Snapchat:
Instagram:
Github:
Discord:
Patreon:

Gear
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
– Video Editing Software (Premiere Pro CC):
– All Adobe Apps (I use this plan):
– Keyboard (Velocifire TKL01) :
– Monitors (Dell U2518D) :
– Webcam (Logitech C920):
– Desk Microphone (Blue Yeti):
– DSLR (Canon T7i):
– Wide Lens (Tokina 11-16mm F2.8):
– Full list of gear:

Music
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Puar – GEMS W/ sin
Link:

bob le head – gapintheclouds [ep] @ 6:52
Link:

dis.joint – knock
Link:

A L E X – Easy Track (part of Growing Up, Vol.2)
Link:

36 thoughts on “Platform for Scalable Web Apps | How I built my website with Kubernetes

  1. Usually with cloud systems is recommended to keep the data that is considered persistent out of the Cluster, so Databases can be Fully Managed Systems that are scalable like Aurora o DynamoDB while files can live in S3/CDN distribution.

    I believe DigitalOcean offers fully managed databases services as well as object storage (kinda S3).

  2. Recommendation…

    Before going into any deployment solution like this… learn linux properly, you are going to use it a lot.

  3. Need a video about your experience with microservices and what advice
    Would you give to someone who is thinking about using microservices.

    Thanks

  4. I can see 3 monitors on your desk… are they connected with the same system or they are separate systems?? If latter, do use a common keyboard + mouse ?

  5. NICE VIDEOS KEEP IT UP. IN YOUR VIDS I SAW MY SELF, LOVE NGINX + CODEIGNITER 4 (BACKEND) + VUE JS (FROENTEND) + GAMESERVERS (WOTLK, L2J), STARTED THINKING ABOUT THAT ARCHITECTURE LIKE YOU (3-6 MONTHS), THAN WATCHED YESTERDAY DOCKET TUTORIAL AND AFTER 10 MINUTES OF WATCHING I FALL ASLEEP AND TODAY I WATCH YOUR VIDEOS ABOUT CS, NGINX ARCHITECTURE AND THIS, AND REMEMBERED THAT I HAVE BEEN WATCHED IT BEFORE, BUT DID NOT UNDERSTAND TOO MUCH, TO BE SURE THAT THIS IS WHAT I EXACTLY LOOKING FOR.. SO YOUR VIDEOS IS THE BEST ONE, I CAN SAY BETTER THAN COMPUTER SCIENCE AT UNIVERSITY, WENT AWAY FROM AFTER 2 YEARS OF WASTING TIME AND MONEY.. THANK YOU AND PLEASE MORE VIDEOS LIKE THOSE!!!!

  6. Where did you learn about container amd kubernetes networking stuff? Is there any one place to have it all?

  7. Watched this video in 2019, now when I came to rewatch it, I understand it better. Maybe I will come again in another 2 years and then understand all of it. MAYBE.

  8. I have the same monitors as you! 🙂 Thanks for the kube video! Curious – Will you please share your monitor calibration settings?! Thinking about getting rid of these because my eyes have been burning for years.

  9. Have you purchased a Kubernetes managed server or have yoh purchased a server and installed Kubernetes in it?

  10. Heyo, just a heads up your site is not pulling data! You're getting lots of 500 errors on the GET requests and uncaught promises 😉

  11. This was great, I'm planning on using Kubernetes for a personal project but found surprisingly free resources for it. It was helpful to see you walk through it

  12. use a kubernetes operator for the database. you can scale it within kubernetes scope, keep trying.

  13. I equally got into micro services by splitting the traditional 3 tier web application into front end , backend, data stores. when you want to modify or add a future, you end up modifying the whole stack which works fine for a personal website or project. Splitting up your web app into services defined by the traditional org charts where frontend, backend, databases are different teams might be an anti pattern in an enterprise scenario. You shouldn’t have a team of frontend devs doing only frontend stuff, you should have teams who own the entire stack for specific functionalities of your web app. That way, they are more autonomous and can scale both the team members and the stack independently of other teams. The micro services teams can choose the best programming languages and data stores for that specific micro service. This is been one of my takeaways from reading Cloud Native Transformations (by Pini Reznik, Michelle Gienow) and Monolith to Microservices by Sam Newman.

  14. Can you do an indepth tutorial on how you approached building micro services? For example how the exact architecture you used look like in addition to what you already explained. That would be most beneficial 😁

  15. I watch what other people do, to feel better of my self, convincing that im doing something which im not.

  16. For the database, you can create a master and slave copy, where you do write operation in master and all read from the slave.
    The other is to split the database based on users
    And, another is some tables in one server, and other independent tables in another server.

  17. Aah the scaling paradox. You’re close. Generally people don’t scale databases that is why the databases are the most beefy part of an architecture. What you can do is though keep a master slave architecture for your database just incase your master dies or you open too many connections by mistake, and you can store an automatic backup of your database in block storage and use that to periodically update your slave database like whenever you make an automatic backup just ask your slave to refresh with the current backup just made. Databases are very safe generally, the code written for mysql has handled all possible situations of data loss and generally you would never see data being lost from a database

  18. More technology doesnt mean better service. In a corporate environment using a million microservices would eventually become a nightmare to maintain. Keep it simple…

  19. Scalable but not large-scale distributed also not that's much secure that's the cons.. Try to work on those area after all it's good

Leave a Reply

Your email address will not be published. Required fields are marked *