Background

The development of rebbl.net started in March 2018.The World Cup Qualifiers were about to start and Cyanide wanted to be able to check the results of the private leagues that were organizing qualifiers. I saw this as an opportunity to finally start a hobby project that actually might be useful.
I had one big decision to make, while I was set on hosting rebbl.net on Azure and building it with NodeJS, the one thing that was going to double the cost was where I was going to store the data, Micrrosoft SQL Server, MariaDB, MongoDB, every form of database required some hosting of its own and would basically double the cost. Until I stumbled upon a little library called NeDB:

Embedded persistent or in memory database for Node.js, nw.js, Electron and browsers, 100% JavaScript, no binary dependency. API is a subset of MongoDB's and it's plenty fast.

Sweet!

It stores it's data in a text file, and upon startup reads everything into the webservers memory and we're set. With a 130 coaches, a maximum of 65 games per round was going to be played for a total of 6 rounds. Looking at how much data a match report consumes, my quick guesstimate was that a total of 6 MB of memory would be sacrificed to the in-mem DB. Noice.


April 4th 2018 was the start of ReBBL's WCQ and rebbl.net was just use for displaying the standings and match results, which looked nothing more than this:

This screenshot is taken from the Wayback Machine and dates back to the 7th of May 2018.

Some of you more sharp-eyed readers might notice that there's links to BIG O, GMAN and REL in that screenshot as well. During the first round of the WCQ the site ReBBL used weekly for posting the current matchups/fixtures (BB2LeagueManager) went down, if I recall correctly it's domain name was expired and took a couple of days to get back up.
This presented me with an opportunity to extend the functionality of rebbl.net to our main leagues as well (as I promised Metal I would do when I came on board as an admin in Season 6). So with just some additional code changes, rebbl.net was set to be hosting the main league standings as well. And on April 17th, 2018, the first scheduling post using rebbl.net was made and all was good.

Queue December '18/January '19. A lot has changed for rebbl.net. We now have:

Yes, I have been a busy boy:

commit history for rebbl.net.

 

Show me the graph, dammit!

Mostly all of the above feature involve saving some data. The largest majority of that is of course from all the matches played, currently rebbl.net has over 11k match records. Which comes down to around 300MB in just match data... in a text file .... in a text file, a  text  file?? Well, not really one, I quite soon realized at the start that everything in one text file wasn't going to work, and I basically partitioned the data over several files, and keeping track of which match report is stored in which file, and only loading those files into memory when needed, and removing them from memory when I was done with them.

This finally resulted in this situation:

That graph show the CPU usage for the past 30 days at 5 minutes intervals on rebbl.net. All the high peaks are the 12 hourly sync of teams. This was changed in November from every 2 hours to every 12 hours. Every 2 hours, for 20 minutes, the website had roughly 90% CPU usage, just syncing teams.
1a: here the sync was completely turned off, because well, I had forgot to turn it on
1b: here the team sync was turned off because the Cyanide API was experiencing issues.
 

Queue MongoDB

I had been toying with the idea of moving to MongoDB since at least September/October last year. I actually set up a VM with MongoDB by the end of October, and did a lot of work on changing the code to make rebbl.net work with MongoDB during November. And then I was stuck. I could not figure out how to go about with going live with all these changes without risking disrupting rebbl.net too much (hence the gap in commits in December).

Then, the week prior to point 2 in the graph (February 22nd, 21:00 UTC) I had worked on getting the Cripple Ladder on rebbl.net. I realized I could not add yet another flat file storage. So I implemented it with the data storage in MongoDB. Which made me realize the way I had set up thing in rebbl.net allowed for me to gradually move data to MongoDB. Thursday and Friday evening (21st/22nd) I just did that, I changed the code for the team and player "service" so it would use MongoDB instead of NeDB. I would like to point out here, that one of the main reasons for picking NeDB was [NeDB's] API is a subset of MongoDB's, which allowed for a very easy transition for these parts of the code. 

The result was much better as I expected, the overall CPU usage went down (you can see that by the height of the "body" of the data-points before and after Point 2), and the big peaks in the graph were cut in half as well (the team syncs). So I let this run over the week while I redid the work from November for the matches and schedules/contests. Those changes got introduced the night of the 27th of February (point 3 in the graph). Now there was just some loose data left, sessions, signups, upcoming games etc. Those were migrated the night of the February the 28th (point 4).

Just wanted to show a bit better what happened at point 3 &4:

i always wanted to do one of these!

My interpretation here:is that moving the match data to MongoDB cut the average CPU usage by another 25% (the length of the pink lines are equal). But moving the sessions to MongoDB had a bigger impact, because while the "body" of CPU usage stayed the same, the overall/baseline usage dropped below 10%. Here's a graph of the past 3 days (just after Point 3 at 1 minute intervals:

Compared to 3 days in January:

just look at those lovely sync peaks!

Shortly after everything was migrated to MongoDB, it struck me. On why everything was so much better CPU-wise. All the time while using NeDB I was focussed on the memory usage of the website. The CPU usage I just attributed to me writing shoddy code, and I could improve that when needed.I felt a bit like an idiot when I realized that when using an in-memory database, it needs to spend CPU on searching in the memory as well, dôh! 

With that in mind, if you compare both 3-day graphs, the peaks in the one from the past 3 days, yeah, that's probably my shoddy code ;)

Little side note, because I don't know where exactly to put this after writing all of this. Size on disk before migration: 350MB, size on MongoDB: 110MB

 

Nice majorbyte, but what does all this mean?!?

Basically it means a huge bottleneck is gone, which in turn opens up for a whole lot more, there's a list of things I want to do on the #rebbl․net-backlog channel on our discord server. Apart from that, it makes the way free to start implement Clan stuff and getting all of the prior rebbl seasons on rebbl.net (some archaeology required), without putting too much of a strain on the website.Also, for those willing to help to pitch in, it should be easier to help out.

Then, there's huge list as well I received from Harringzord, with all the feedback from the Nufflies on rebbl.net. I will soon update the backlog on discord with this feedback.

 

Finally, I would like to thank you all for the donations I have received so far.

It's always nice to have something to drink while doing this!

Having said that, and having mentioned cost for hosting earlier in the post, I feel I have to emphasize that the actual cost of running rebbl.net is fuck-all at the moment, because reasons. I do know what the actual hosting costs in azure are, and have always strived to keeps those down in case the fuck-all reasons disappear. So donations are solely for the Majorbyte Beer Fund and nothing else.

- majorbyte

Read more from this writer.

Comments

I have no idea what any of this means but I think you're telling me our world domination plan is on schedule - Harringzord

it is indeed - majorbyte

this has something to do with the second coming? - Incesticide

I am a total Philistine. Only two things came to my mind: "Ohh, shiny colors" and "Looks like Majorbyte is actually working all the time on the website". Kudos, seriously. - Zee

Thanks for the insight and for all you do! - Larkstar

you are a saint - Rusty Seabutter

Log in to add a comment

Comments

I have no idea what any of this means but I think you're telling me our world domination plan is on schedule - Harringzord

it is indeed - majorbyte

this has something to do with the second coming? - Incesticide

I am a total Philistine. Only two things came to my mind: "Ohh, shiny colors" and "Looks like Majorbyte is actually working all the time on the website". Kudos, seriously. - Zee

Thanks for the insight and for all you do! - Larkstar

you are a saint - Rusty Seabutter

Log in to add a comment

More REBBL Content

These links lead to other REBBL related content, information, and websites.

Latest News Posts
REBBL Reddit Homepage
REBBL Match Ups and Stats
Nufflytics - Data Analysis
How to Write for REBBL News

Huge Jackedman Memorial Cup 6: Return of the Jack - Signups Open

Gerbear's Announcement of the next tournament that occurs on Saturday, June 1st, starting around 8:00 UTC.

It’s Podcast Recap Day My Dudes!

A recap of the latest episode.ep. 78

Taproom Sessions - Seventh Pitcher

Yvresse, quand tu nous tient. A Bretonnian in Bordeleaux. A well respected man about town. Local handpicked heroes triumph over bloodsuckers.

Taproom Sessions - Sixth Pitcher

Perks of Vampire coaching. Drifting to Tokyo. The bawdy tale of Krumm-Huegel. Cold Norse front hits Tempelhof.

No Fangs Week 5 & 6 Progress Report

We share some sad, sad news