Why I decided to write my own media hosting service in Vue and Node.js
For fun in my spare time, I’ve been writing an image and media hosting service for myself and my wife. I’ve also been contemplating open-sourcing it for those who may want to host it themselves, or even potentially creating a Saas for people who want such a service but aren’t interested in self-hosting.
The project is currently a functional but rough prototype. I plan to create a working web app in Vue and eventually an iOS and Android app in Flutter by the beginning of next year.
Where did this idea come from and why would I attempt such a thing?
Good question. In light of the past few years regarding data privacy and data security, I have increasingly been moving my data (files, media, etc.) off of large cloud platforms such as Google Drive and the like, in favor of self-hosting my data. See below resources.
Google accused of leaking personal data to thousands of advertisers | ZDNet
Brave, the maker of a Chromium-based browser rival to Google Chrome, says it has new proof that Google is leaking…
5 Million Google Passwords Leaked
Google users are being urged to change their passwords in the wake of 5 million stolen credentials surfacing on Russian…
Hundreds of millions of Facebook user records were exposed on Amazon cloud server
More than 540 million records about Facebook users were publicly exposed on Amazon's cloud computing service, according…
Self-hosting has worked far better than I could have expected due to an increase in internet bandwidth, faster computers and mobile phones, better networks, and many affordable self-hosting solutions.
With that in mind, I found Nextcloud. This particular solution seemed to meet most of my needs. After using Nextcloud for almost a year, I can say that in a lot of ways it’s just as reliable as Google Drive and has many of the same features you would expect. However, I would also add that the UI and desktop apps are fairly dated. Their UI and some of the syncing functionality is questionable at best. For instance, the menu bar on Chrome has been broken for over a year.
I also noticed that the app and UI are unresponsive or extremely slow for large amounts of pictures and media. Finally, the kicker is that everything is written in PHP. I had thought about contributing to the project, but PHP was a major turn-off for me. I really appreciate their efforts and what they bring to the community, but for me, the hit and misses of the product left both me and my wife wanting a better solution for us. That is where Eikona comes into play.
Currently, without any major optimizations, I’m able to load 2,500+ photos almost instantly without any caching. As I continue to add optimizations including streaming of images and infinite scrolling (a form of pagination), I expect that Eikona should easily be able to load photo collections on desktop and mobile with 100,000+ photos without buffering or waiting for thumbnails to load.
Right now, the only optimization that’s done is reducing the size of each image by creating thumbnails that are 250x250. That makes them about 1/10 the size of the original file. This allows, at least locally, thousands of thumbnails (photos) to load almost immediately. When you click the thumbnail, the full resolution file loads. A 12 kb thumbnail then loads a 120kb photo. Even for large JPEGs, the thumbnail is 12kb while the actual photo may be 705kb+.
New updates coming soon including a post on how I maxed out the new Apple silicon M1 processor and 16GB RAM while doing image processing. I will post my results with code, screenshots, and optimizations to improve the processing of tens of thousands of photos.