May 16, 2018 at 2:49 pm #7322
Hello, just letting everyone know I’m still here and working on the game daily. I’ve been making progress on several things, mostly Wellington so I can release that for public testing soon. The map north of Market Square and the wolf cave should also be in the game shortly.
The population has increased a lot, and the servers have been handling it pretty well thanks to the stability and lag optimizations from earlier this year. If the game keeps growing, the next server will be another east coast US server.
There are still some small bits of lag occasionally, and a fairly rare server crash but it has been acceptable. Though when servers stay up for a long time the lag becomes more noticeable. So I’ve added a feature that lets me send out a graceful reboot command (most likely this will happen late at night). This will give periodic warnings before restarting a server automatically.
Other than that I’ve noticed that Brazil (which is on a different data center from the other servers) seems to have occasional issues that I don’t have control over.. but hopefully those will be resolved soon if they aren’t already. If the problems continue then I may have to consider moving Brazil closer to the states, which means a higher ping but better network stability overall.
Anyways I’m hoping to have some new content out by next week or earlier. I’m also looking forward to releasing some other fun changes, like new gem bonuses. Thanks for all your support!May 25, 2018 at 7:00 pm #7622
The new east coast server is live! For lack of a better name it is simply USEast2. This particular server is more costly and should perform better. So if the game keeps growing I’ll start upgrading more servers to this kind to keep gameplay smooth and alleviate lag for older servers.
The Wellington patch is going well. I made some adjustments since then: SK Boss can heal while the room is empty, and doesn’t give as much myst to weak characters. Channeling water no longer works on the edge of maps, and digging in water can occasionally cause adjacent water to become shallow. I also put in some code to fix damage on old hammers.
Will be keeping an eye on things. If you see any bugs or know of anyone blocking off houses in Wellington please submit a contact form message. Thanks, and have fun!May 26, 2018 at 2:24 pm #7653
Just an update on the server situation. Over the last few days the data centers I’ve been using have been suffering more and more from noisy neighbor issues.. stealing our CPU cycles, especially on the US servers.
I mentioned in my last notes that I’m testing new hardware with USE2. I’ve also been testing this new kind of server for a few days with BR, which regularly has over 100 unique players. The performance is definitely better on the new servers, and so far there is no CPU steal.
So I think it’s worth the costs to upgrade/migrate all the servers. I’ll be doing that asap for the old servers, starting with the US based ones which are having the most issues. I’ll also try to spin up new servers more rapidly if growth continues, as that always alleviates the load on each server. Thanks for your patience and support!May 27, 2018 at 5:07 pm #7695
Another update on server growth and stability:
I’ve added a new SA server since there are so many players coming from South America it will be nice to have a couple options. Plus we need more room to grow, as we keep topping our record for players online!
Last night I migrated the US servers to new hardware which reduced lag by quite a bit. I still have to migrate London, EU, and SEA which will probably be complete within the next day or two.
I noticed that USW lag had the biggest reduction – but USE and UST only had a moderate improvement. I’d like to get that better so it’s on par with the other servers.
I’ll get a bit technical to explain this. Basically the garbage collection on those servers is having some hiccups. It’s what causes the occasional rubber banding. The code behind that is not under my control, there are two other teams that manage that process and I’ve been following their progress.
So I’m testing a new version of Node (the code that runs the server) and I’m seeing some promising results. Basically when we were running on Node 8 earlier this year it caused large and progressively increasing delays when trying to reclaim memory.. some of you might remember how bad it was back then. I thought it was my code and spent two months optimizing everything, which has been great, even though it didn’t solve the problem I was looking for at that time.
I found out that going to an older version, Node 6, actually made the pauses unnoticeable most of the time. That’s what we’ve been using for the last few months which has been pretty decent – but it could be better. Version 6 is feeling less efficient as the servers grow larger. I’ve tested versions 4,7,8,9.. it’s all bad. Versions 10 and 11 seemed great but were buggy for a while. Now they seem to be working better, and the garbage collection performance is amazing so far. If testing continues to go well with 10+ and they’ve fixed the bugs then I will start switching some servers over to this new engine soon, and hopefully those annoying rubber bands will be behind us 🙂
Also, one quick change which is live – I made it so building fires in Wellington costs a lot less myst than other structures.
Since the patch a few days ago, across all servers there has only been one reported case of Wellington griefing that required admin intervention. It only took a few seconds to fix the issue, so I may not have to give Wellington citizens any extra tools after all. I’ll still keep an eye out to make sure everything is balanced and manageable. Thanks for your feedback, I’ll post another update soon 🙂May 29, 2018 at 3:05 pm #7757
Here’s the latest:
-Fixed a bug where towers wouldn’t give infamy to a raider if they recently hit an infamous player
-Made the depth recall only work from underworld maps, as it was before the new maps were added
Thanks to Therieth for the detailed reports on those. Kay is investigating server logs related to the past reports on the tower issue. Feel free to send any more details through the contact form.
I’ve been testing different types of servers with different versions of Node over the last couple of days. 6 is still the most stable, and a few servers are running on it now. 10 is still very unstable and no servers are running on it, but 11 seems to run well for several hours with minimal pauses for cleanup. Sometimes there was no server lag at all for long periods of time, which was awesome to see. It loads up faster as well, but I did notice increasing instability over time with issues arising on SEA and USE especially. So unfortunately, until their code is more mature I’m going to be reverting every server back to 6.
The lag on the new hardware is still 2-3 times less frequent than it was previously. Currently It’s about 1/3rd of a second pause every 5 minutes or so, under heavy load. On version 11 that mini-pause happens about once an hour, but then over time the pause duration can increase up to 2 seconds(!), among other issues. So we’ll probably be fine on 6 while we wait for updates.
I’ll keep a close eye on that, but now that the population is better spread out I’ll shift to working on some new balance and content patches. There are a couple new accessories coming, along with maps, and costumes. I’m going to implement some exclusivity for the better looking costumes soon, to reward big supporters. That may help raise funds to add and upgrade servers even further, anticipating the population increase from the future IOS release. Hopefully I don’t get rekt by summer break before then 😀May 30, 2018 at 4:05 pm #7772
-Underground walls can now appear on dirt floor, without being adjacent to other dirt walls
-Now when the negative water digging event occurs it keeps the water shallow, rather than going straight to dirt, and adjacent water has a 50% chance to go shallow rather than 100%
The topic ‘May Dev Notes’ is closed to new replies.