The development of rebbl.net started in March 2018.The World Cup Qualifiers were about to start and Cyanide wanted to be able to check the results of the private leagues that were organizing qualifiers. I saw this as an opportunity to finally start a hobby project that actually might be useful.
I had one big decision to make, while I was set on hosting rebbl.net on Azure and building it with NodeJS, the one thing that was going to double the cost was where I was going to store the data, Micrrosoft SQL Server, MariaDB, MongoDB, every form of database required some hosting of its own and would basically double the cost. Until I stumbled upon a little library called NeDB:
It stores it's data in a text file, and upon startup reads everything into the webservers memory and we're set. With a 130 coaches, a maximum of 65 games per round was going to be played for a total of 6 rounds. Looking at how much data a match report consumes, my quick guesstimate was that a total of 6 MB of memory would be sacrificed to the in-mem DB. Noice.
April 4th 2018 was the start of ReBBL's WCQ and rebbl.net was just use for displaying the standings and match results, which looked nothing more than this:
This screenshot is taken from the Wayback Machine and dates back to the 7th of May 2018.
Some of you more sharp-eyed readers might notice that there's links to BIG O, GMAN and REL in that screenshot as well. During the first round of the WCQ the site ReBBL used weekly for posting the current matchups/fixtures (BB2LeagueManager) went down, if I recall correctly it's domain name was expired and took a couple of days to get back up.
This presented me with an opportunity to extend the functionality of rebbl.net to our main leagues as well (as I promised Metal I would do when I came on board as an admin in Season 6). So with just some additional code changes, rebbl.net was set to be hosting the main league standings as well. And on April 17th, 2018, the first scheduling post using rebbl.net was made and all was good.
Queue December '18/January '19. A lot has changed for rebbl.net. We now have:
- Side leagues: Excessively Elfly, Lineman League, One Minute Showdown, the Open Invitational, the Greenhorn Cup, Eurogamer Open Invitational.
- We have new structures within the league, 10-coaches-divisions with an additional 2 round Swiss that count towards the standings, and Playins (purposefully not linking Playins, because the link is broken 🤭)
- We have RAMPUP for GMAN and REL.
- Playoffs are on the site as well
- Team and player statistics
- Upcoming games with the possibility for streamers to tag games they are going to stream.
- Signups not to forget (anyone remember these? ) accompanied with logging in through reddit, and the required https certificates for that.
- And other features, like being able to see your upcoming opponents,
Yes, I have been a busy boy:
commit history for rebbl.net.
Show me the graph, dammit!
Mostly all of the above feature involve saving some data. The largest majority of that is of course from all the matches played, currently rebbl.net has over 11k match records. Which comes down to around 300MB in just match data... in a text file .... in a text file, a text file?? Well, not really one, I quite soon realized at the start that everything in one text file wasn't going to work, and I basically partitioned the data over several files, and keeping track of which match report is stored in which file, and only loading those files into memory when needed, and removing them from memory when I was done with them.
This finally resulted in this situation:
That graph show the CPU usage for the past 30 days at 5 minutes intervals on rebbl.net. All the high peaks are the 12 hourly sync of teams. This was changed in November from every 2 hours to every 12 hours. Every 2 hours, for 20 minutes, the website had roughly 90% CPU usage, just syncing teams.
1a: here the sync was completely turned off, because well, I had forgot to turn it on
1b: here the team sync was turned off because the Cyanide API was experiencing issues.
I had been toying with the idea of moving to MongoDB since at least September/October last year. I actually set up a VM with MongoDB by the end of October, and did a lot of work on changing the code to make rebbl.net work with MongoDB during November. And then I was stuck. I could not figure out how to go about with going live with all these changes without risking disrupting rebbl.net too much (hence the gap in commits in December).
Then, the week prior to point 2 in the graph (February 22nd, 21:00 UTC) I had worked on getting the Cripple Ladder on rebbl.net. I realized I could not add yet another flat file storage. So I implemented it with the data storage in MongoDB. Which made me realize the way I had set up thing in rebbl.net allowed for me to gradually move data to MongoDB. Thursday and Friday evening (21st/22nd) I just did that, I changed the code for the team and player "service" so it would use MongoDB instead of NeDB. I would like to point out here, that one of the main reasons for picking NeDB was [NeDB's] API is a subset of MongoDB's, which allowed for a very easy transition for these parts of the code.
The result was much better as I expected, the overall CPU usage went down (you can see that by the height of the "body" of the data-points before and after Point 2), and the big peaks in the graph were cut in half as well (the team syncs). So I let this run over the week while I redid the work from November for the matches and schedules/contests. Those changes got introduced the night of the 27th of February (point 3 in the graph). Now there was just some loose data left, sessions, signups, upcoming games etc. Those were migrated the night of the February the 28th (point 4).
Just wanted to show a bit better what happened at point 3 &4:
i always wanted to do one of these!
My interpretation here:is that moving the match data to MongoDB cut the average CPU usage by another 25% (the length of the pink lines are equal). But moving the sessions to MongoDB had a bigger impact, because while the "body" of CPU usage stayed the same, the overall/baseline usage dropped below 10%. Here's a graph of the past 3 days (just after Point 3 at 1 minute intervals:
Compared to 3 days in January:
just look at those lovely sync peaks!
Shortly after everything was migrated to MongoDB, it struck me. On why everything was so much better CPU-wise. All the time while using NeDB I was focussed on the memory usage of the website. The CPU usage I just attributed to me writing shoddy code, and I could improve that when needed.I felt a bit like an idiot when I realized that when using an in-memory database, it needs to spend CPU on searching in the memory as well, dôh!
With that in mind, if you compare both 3-day graphs, the peaks in the one from the past 3 days, yeah, that's probably my shoddy code ;)
Little side note, because I don't know where exactly to put this after writing all of this. Size on disk before migration: 350MB, size on MongoDB: 110MB
Nice majorbyte, but what does all this mean?!?
Basically it means a huge bottleneck is gone, which in turn opens up for a whole lot more, there's a list of things I want to do on the #rebbl․net-backlog channel on our discord server. Apart from that, it makes the way free to start implement Clan stuff and getting all of the prior rebbl seasons on rebbl.net (some archaeology required), without putting too much of a strain on the website.Also, for those willing to help to pitch in, it should be easier to help out.
Then, there's huge list as well I received from Harringzord, with all the feedback from the Nufflies on rebbl.net. I will soon update the backlog on discord with this feedback.
Finally, I would like to thank you all for the donations I have received so far.
It's always nice to have something to drink while doing this!
Having said that, and having mentioned cost for hosting earlier in the post, I feel I have to emphasize that the actual cost of running rebbl.net is fuck-all at the moment, because reasons. I do know what the actual hosting costs in azure are, and have always strived to keeps those down in case the fuck-all reasons disappear. So donations are solely for the Majorbyte Beer Fund and nothing else.