Jump to content
ALTIS LIFE
...
GTA RP
...
TEAMSPEAK
...

We are currently undergoing maintenance

The forums are available for use, yet please do not expect full functionality.

Ciaran

Altis Life UK Server Development - Feedback

Recommended Posts

TikTak

Had some mass kickes after latest update

  • Like 2

Share this post


Link to post
Ciaran

The network update was to reduce desync, the mass kicks are being looked into now.

  • Like 7

Share this post


Link to post
Dan B

Can't join the server 'connection failed'

Share this post


Link to post
HercEU

The server stability is getting worse and worse. It started with a few random kicks then a lot more of those then, now, we have mass kicks that empty the server up. Even more so for the past few days I and some of my friends are unable to even connect, getting "Connecting failed" messages when the server is clearly online. And to top it all off staff's attitude towards all of this is "it'll get fixed when it gets fixed". Not sure that's a good attitude towards the community but what do I know. Anyway, those are my 2 cents.

  • Like 2

Share this post


Link to post
La Crox

Is it possible just to go back to 130 slots?

the problems all started from there and i think i speak for everyone if i say that we wouldn't mind 10 slots so we could have fun on the server.

  • Like 4

Share this post


Link to post
Smith

When someone says it will get fixed when it gets fixed that means that we cannot give you a concrete time on when something will be fixed. Doing so would likely lead to you have false hope over a promise we cannot give you. Our team works tirelessly around the clock, if we have fixed a problem we dont sit on it and release it only when we can be arsed, we fix things as thoroughly as we can, and yes, when a tested fix is ready, it will be released. If there's not a fix out it's either being worked on or tested, we do t withhold things from you.

  • Like 3

Share this post


Link to post
Ciaran

The kicking issues are being investigated, we are collecting as much information as we can about the kicks, if you can't get on S1 at the moment then please use S2.

  • Like 3

Share this post


Link to post
Banci
4 hours ago, La Crox said:

Is it possible just to go back to 130 slots?

the problems all started from there and i think i speak for everyone if i say that we wouldn't mind 10 slots so we could have fun on the server.

They didnt start there but it didnt get any better.

  • Like 1

Share this post


Link to post
Terry Sugarbumps

I don't know if this is the right place for this but Bohemia are releasing a free map dlc this year called "Malden". Seeing that this map is free unlike Tanoa. Maybe server 3 could come into good use 

  • Like 2

Share this post


Link to post
Thor God of Thunder

It might just be me but it seems that the random kicks have gotten worse of late. I got randomly kicked 4 times in under an hour today, which is significantly more than the normal once or twice every four hours. Thank you for your time, I am sure others have posted this problem as well, so I will avoid being a pain in the ass

  • Like 1

Share this post


Link to post
Papa John

I think the airdrops do not have a place in a roleplaying server especially if you can get 7.62 LMG'S there

Just Telling my opinion

  • Like 2

Share this post


Link to post
Simen
8 minutes ago, Tuplajuusto said:

I think the airdrops do not have a place in a roleplaying server especially if you can get 7.62 LMG'S there

Just Telling my opinion

Well given only two exist on the island at this very moment. I don't see too much of a problem. Especially when one of them is locked up in a police officers house waiting for its owner to go on his/her next police holiday.

  • Like 1

Share this post


Link to post
Mr Cardoso

Doesn't it all depend on the owner of the gun? I mean the airdrop system is not closed and can always be tweeked / changed.

But if you have a big gun or not, you only use if it you want.. you can even sell it :) #roleplayeverything

  • Like 1

Share this post


Link to post
Guest
1 hour ago, Mr Cardoso said:

Doesn't it all depend on the owner of the gun? I mean the airdrop system is not closed and can always be tweeked / changed.

But if you have a big gun or not, you only use if it you want.. you can even sell it :) #roleplayeverything

Regarding the airdrop I have a few statements i'd like to make:

Currently the airdrops spawn everywhere on the map. Personally, I feel they should only drop in redzone locations to keep them confined away from possible rp that could be occuring in other areas. Equally, 200M around the airdrop is too small. What it leads to is people staying outside the redzone area and running around it and only entering once they see someone they can easily frag etc. If you was to put it into redzone this wouldn't be the case

Airdrops are well to easy to swipe. Once it swaps you can literally just go to it, take the gun and be gone instantly. The idea of the airdrop is good however how it's been implemented I personally disagree with. 

My recommendations for this are: 

Make it so you need bolt cutters in order to get into it. This will stop people getting to it and instantly taking it without even any combat. They're dropping rare items and as a result should be harder to take. 

Personally, I think once it's been announced it should have a 5min delay timer. What I mean by this, is the announcement will appear in the top right saying 'An airdrop is imminent at xyz'. Once this is said it gives 5 minutes for players who want to get the airdrop to head over to the location. When the 5minuties is up, the airdrop will fly down. With the implementation of this, it will lead to there being more people actually contesting this air drop idea because as of right now you can just get really good loot super easily.

My final suggestion is to personally increase the spawn time of it. As it's a new feature in the game I have been waiting the whole server restart for these things to go off. Currently, when the server restarts nothing happens for 40mins, then when the 40mins are up something will spawn but it's not always the airdrop. This means I need to wait 30mins for this to despawn and another 30mins for a new objective to re-appear. Sometimes i've gone full restarts without even seeing one airdrop on S1 at peak time.

If you were to alter the system into what i've suggested it would be much more enjoyable as you'd have squads of gangs waiting for the airdrop to commence and once it does getting to the position and holding the area until the airdrop drops. 

I really hope you consider my ideas which should be very simple to implement.

  • Like 2

Share this post


Link to post
LukeN
1 hour ago, GHOSTK1LL3R said:

Regarding the airdrop I have a few statements i'd like to make:

Currently the airdrops spawn everywhere on the map. Personally, I feel they should only drop in redzone locations to keep them confined away from possible rp that could be occuring in other areas. Equally, 200M around the airdrop is too small. What it leads to is people staying outside the redzone area and running around it and only entering once they see someone they can easily frag etc. If you was to put it into redzone this wouldn't be the case

Airdrops are well to easy to swipe. Once it swaps you can literally just go to it, take the gun and be gone instantly. The idea of the airdrop is good however how it's been implemented I personally disagree with. 

My recommendations for this are: 

Make it so you need bolt cutters in order to get into it. This will stop people getting to it and instantly taking it without even any combat. They're dropping rare items and as a result should be harder to take. 

Personally, I think once it's been announced it should have a 5min delay timer. What I mean by this, is the announcement will appear in the top right saying 'An airdrop is imminent at xyz'. Once this is said it gives 5 minutes for players who want to get the airdrop to head over to the location. When the 5minuties is up, the airdrop will fly down. With the implementation of this, it will lead to there being more people actually contesting this air drop idea because as of right now you can just get really good loot super easily.

My final suggestion is to personally increase the spawn time of it. As it's a new feature in the game I have been waiting the whole server restart for these things to go off. Currently, when the server restarts nothing happens for 40mins, then when the 40mins are up something will spawn but it's not always the airdrop. This means I need to wait 30mins for this to despawn and another 30mins for a new objective to re-appear. Sometimes i've gone full restarts without even seeing one airdrop on S1 at peak time.

If you were to alter the system into what i've suggested it would be much more enjoyable as you'd have squads of gangs waiting for the airdrop to commence and once it does getting to the position and holding the area until the airdrop drops. 

I really hope you consider my ideas which should be very simple to implement.

Put this in suggestions mate

  • Like 1

Share this post


Link to post
TinyBigJacko

Some feedback on the 'mass kicks' and 'random kicks' issues.

Firstly, be assured that we are working on this pretty much fulltime at the moment. I am doing virtually nothing else right now, and have so many windows open on my various development machines, that I am beginning to think I will never see sunlight again.

We have two ongoing problems, as far as we can tell.

The first, the 'kicks with no reason' or the 'random kicks' is the situation where a player gets kicked off, and is given the simple, rather blunt message "you were kicked off the game". No other information, no mention of admins, Battleye, or anything approximating a reason, just 'you were kicked off'. Really annoyingly, our RCON system logs this purely as a 'player disconnected' (which of course, he or she did, but not voluntarily!), and mentions nothing about it being a kick. Even the ARMA3 server log records the same - a 'player left' notification, with no hint of any kick or kick-reason. It is totally inseparable and indistinguishable from a normal 'player leaving' notification, which of course makes debugging virtually impossible (because players genuinely leave the game hundreds of times a session - people come and go).

Our only current angle with debugging these (from an 'after-the-fact' perspective) is the realisation that in-game, other players will see the message 'Player X was kicked off the game' (or something similar) and then the 'player X disconnected' message below it, all in the system-chat on the lower left-hand side. This is our only clue that this was a 'random kick' and not just a player leaving of their own volition.

What I'm trying to do currently is gather many of these incidents via a packet-sniffer running on the server, by re-running and recycling a capture-buffer, until one of these reports comes up in system-chat. Then I save the capture, take notes about the player's 'drop' time, their IP address, and suchlike, and then set out analysing their packet stream in the moments preceding their kick, to see if there is anything obvious.

The only fly in the ointment, of course, is the fact that Bohemia have encrypted the UDP packet-data in order to stop people injecting cheats into the data-stream, and this currently means that all the packets we capture are effectively indecipherable, in terms of working out exactly what the server and the player's client thought was being 'discussed', in game-data terms. We're currently trying to source a method of decoding these packets reliably, in case it sheds any light on the problem, but of course, finding this information out, and a Wireshark ARMA3 UDP packet-decoder (one which actually works and is current), involves visits to the 'dark side', which is of course not something we're very keen on. Steps are being taken, though...

Meanwhile we're still gathering data in the hope that it will give us something to go on, simply by virtue of the data-sequences that we see repeated, during and up to a random, no-message kick. Patterns still emerge, even if the precise detail is obscured, sometimes.

Then there is the 'mass-kick' problem... at first, we thought these were due to our own game script code, maybe something we'd added in a recent update. We have pulled out all sorts of features over recent days, in the hope of stopping the problem. We tuned down audio codecs (resulting in loads of people complaining about poor audio), we turned off weather, we tweaked vehicle management, altered loads of Battleye script detections, and, well, did too many things to list, really. Obviously this is an extremely slow and laborious process - turn off one thing in the development repo, wait for a server restart window, update the mission-code on the server, restart, sit back and wait, and hope that the kicks have gone. Rinse and repeat when we discover that they haven't gone, and came back when the server was next fully populated (which might be 24 hours away from the last update, depending on the time!) Then reset everything and try again. Mind-numbingly dull and frustrating, when it doesn't resolve the issue solidly. Especially when the mass-kicks sometimes seem to vanish for a couple of days; maybe due to population levels, maybe due to some person simpy hitting 'the magic trigger' in game (whatever it turns out to be). We don't know yet... but we will.

Worse, with the 'mass-kicks' we are unable to grab a complete packet-dump during the time of the kicks, because the server is flooding out many megabits of data per second, and it blows up the protocol analyser's capture-buffer, and leaves us with nothing. Totally frustrating. Many rude words have been said and invented over recent days, specifically due to this. Seeing the packet storms, we wondered if we were being DDOSed, but not getting reports from our server host. We actually changed to a different server host in order to run some comparative tests, and then we discovered (by some helpful input from our host) that the packet-storms were being generated internally, and not coming from outside at all! Basically our own server was going into 'scream mode' - and in the process, deafening itself, and shutting out Battleye. This in turn results in large chunks of the player-base losing contact with the server, which Battleye interprets as a 'no response'... and eventually, it's Battleye that does the kicking. Plonk! Anywhere from 3 to 100 players all get kicked with Battleye: Client Not Responding at exactly the same time - which itself, may actually be several minutes or seconds after the actual source of the problem (such is the nature of timeout periods triggering).

As you can imagine, we've tried a hundred different firewall configurations, server config tweaks, bandwidth filter modifications or removals, all to no avail, so far. I have a hunch that maybe the 'random no-message kicks' are possibly related (maybe piling up at certain points, and bottlenecking the server or something), but it is just a hunch, really. We'll know more when we are able to conduct some real in-depth packet-data analysis, I hope.

One thing I did discover is that this has been going on a long time. Much longer than I imagined, and the effect appears to have increased recently. I conducted some data-analysis on our ARMA3 logfiles, gathered since last August, and scoured every single one for all mentions of 'Battleye Client Not Responding' kicks. Obviously some of these will legitimately be just people getting kicked for having crummy network ping-rates or lossy streams... but it was still a valid exercise. I was able to pinpoint the first date where we saw multiple people all being kicked for 'Battleye Client Not Responding' at the exact same date, hour, minute, and second. And from that date, track an overall rise, to the current date.

The date it all seems to have started was 11th October 2016 - the date we released our big Altis update, which pulled in a great many features from the Tanoa build. Of course, our immediate thought was that it was some of the Tanoa core code, and we've set about disabling (temporarily) various features that might be an issue, or rewriting certain loops and routines where it's essential we keep that code in place, in such a way that they cannot 'lock' the server for any length of time. Not that we're certain this is happening, but it's good practice, and development is an iterative learning process, so we've learned plenty even since then, plus we've had new people like Rath join us, bringing ever more knowledge to the table.

However, it turns out it may not even be our code after all. The dice are still rolling. On the 11th October, Bohemia Interactive released a Hotfix to the game-engine. I remember it well; Wilco and myself laughing at the timing, because it happened literally minutes before we were holding down the servers in order to update the mission-file, and we knew full well that this was going to make for a muddied audit-trail in the future, if ever anything went wrong. We'd never know whether the fault was ours, or Bohemia's... and now here we are six months down the line, wincing and laughing wryly, as we look back at that date. 

I'm not going to say at this point, that this is an ARMA problem. It might be. It might not be. We simply do not know yet. But we do know (as of only a day or two ago) that we are not alone in suffering it, and this is a possibly crucial turning point. We have some friends in other large ARMA communities - some running Life games, some not, plus we have some not-so-friendlies in other communities, and they make server reports just like this one, from time to time. Turns out that some of the communities running larger-population servers are suffering the exact-same 'mass kick' Battleye Client Not Responding problem as we are. Identical. We're currently in the process of trying to establish whether certain specific, recent (as of October last year) enhancements to the ARMA scripting are being used on their games, as they are on ours - things like SimpleObjects, for example, and whether they are also suffering from the quiet-but-constant residual 'no-message random kicks' too. 

We're hopeful that we can find the root cause, and nail it. Maybe by information-sharing with a few other cool and clever people in the wider community, we can move faster, but in any event, we're still doing all we can to get to the bottom of it. Admin staff will, of course, just tell you that everything which can be done is being done, and they won't have any answers about dates of fixes, or reasons, nor will they want to hear you cussing them out, or cussing out the dev-team or management, for fairly obvious reasons. They're busy trying to hold it all together during this process, and continue their job of managing the players and ensuring fair play. Please support them in this objective, and realise that they are not being 'indifferent' or 'hostile' by not telling you exactly what's going on. They're doing their job as well as they can, in the circumstances.

However, I wanted to make this post just so that you knew a little bit more about what was going on down here in the BatCave, and what we're trying to do to resolve the problem. Of course I can empathise with you, if you're affected, and losing play progress or gear due to a random kick or being inconvenienced massively during a mass-kick and losing all your fellow roleplayers! What I cannot yet do, though, is give you promises of a date when it will stop. I can promise you that we're doing a great deal of work to get to the bottom of it - work which, if I'm honest, Bohemia really ought to be helping us (and others) to do more easily... but we're well-used to being left out here on a limb at the sharp-end by now, so I'm not going to complain about that too much! It achieves nothing, and we have some strategies in that 'direct-approach' department that we're about to use...

All I can say for now, is sit tight, weather the storm, and keep the faith. RPUK will always do its best for you, the player.

As Liam Neeson (almost) said: "Glitch, I don't know what you are. I don't know what you want. If you are looking for ransom, I can tell you I don't have money. But what I do have is a very particular set of skills; skills I have acquired over a very long career. Skills that make me a nightmare for glitches like you. I will look for you, I will find you, and I will kill you."

  • Like 30

Share this post


Link to post
YoCo
1 minute ago, DSGT Fergus said:

@TinyBigJacko Could you give me a TLDR please <3

They're unsure of exact issue, but they're trying the damnest to fix it ASAP.

Share this post


Link to post
Guest
1 hour ago, TinyBigJacko said:

[Insert big words into big post here]

Just an FYI, considered pushing out the update prior to the tanoa one you were referencing on the cop practice server. See if people not getting kick and if not push out the tanoa one and then start removing the big updates that you implemented.

Gives a good sandbox where there will be players who can tell you if they're getting kick and won't drastically destroy gameplay on that server?

 

p.s. Im sure you've tried everything but is an idea you could try if you haven't considered it.

Share this post


Link to post
TinyBigJacko
2 minutes ago, DSGT Fergus said:

@TinyBigJacko Could you give me a TLDR please <3

Yes, of course.

TL;DR? 

You missed some interesting shit.

1 minute ago, GHOSTK1LL3R said:

Just an FYI, considered pushing out the update prior to the tanoa one you were referencing on the cop practice server. See if people not getting kick and if not push out the tanoa one and then start removing the big updates that you implemented.

Gives a good sandbox where there will be players who can tell you if they're getting kick and won't drastically destroy gameplay on that server?

Sadly, it won't work like that. At the time, we had several versions in parallel - and some that were not released or developed fully (remember I re-joined at the time that Noms left, and his entire codebase was dropped in favour of a return to the older ALUK core we were already running on Altis at the time. It's rather complicated to explain, but trust me, we don't simply have a 'version before'... because so much was done on Tanoa over three months and then folded back in to Altis when we killed off Tanoa.

Also, we must consider that S2 is not nearly so badly affected as S1 - regardless of where S1 has been hosted... so we can only presume that 'load' is an issue here too. Whenever we try and roll out a 'test' version of anything, we are never able to get it fully loaded, try as we might, so there is a chance we'd never get a test server to the point of crunch, even if we put it out there. Really, the only way to be sure of that, is to replace S1 with it, and that is very, very risky.

Hence our approach of selectively 'chopping out' or 'rewriting' bits of the code, and releasing it live to S1, so we can be reasonably sure that it will be loaded to the max within about 24 hours or so. It's not an ideal approach, but it's all we can logically do, in the circumstance. Worse still, we're now not even entirely sure it *is* our code at all, given the significance of the release-dates and the fact that other communities are suffering the exact-same problem.

  • Like 5

Share this post


Link to post
Guest
1 hour ago, TinyBigJacko said:

Yes, of course.

TL;DR? 

You missed some interesting shit.

Sadly, it won't work like that. At the time, we had several versions in parallel - and some that were not released or developed fully (remember I re-joined at the time that Noms left, and his entire codebase was dropped in favour of a return to the older ALUK core we were already running on Altis at the time. It's rather complicated to explain, but trust me, we don't simply have a 'version before'... because so much was done on Tanoa over three months and then folded back in to Altis when we killed off Tanoa.

Also, we must consider that S2 is not nearly so badly affected as S1 - regardless of where S1 has been hosted... so we can only presume that 'load' is an issue here too. Whenever we try and roll out a 'test' version of anything, we are never able to get it fully loaded, try as we might, so there is a chance we'd never get a test server to the point of crunch, even if we put it out there. Really, the only way to be sure of that, is to replace S1 with it, and that is very, very risky.

Hence our approach of selectively 'chopping out' or 'rewriting' bits of the code, and releasing it live to S1, so we can be reasonably sure that it will be loaded to the max within about 24 hours or so. It's not an ideal approach, but it's all we can logically do, in the circumstance. Worse still, we're now not even entirely sure it *is* our code at all, given the significance of the release-dates and the fact that other communities are suffering the exact-same problem.

Good luck then :P

  • Like 2

Share this post


Link to post
Lemmen
1 hour ago, TinyBigJacko said:

Lovely post

What did we do to deserve you? <3

  • Like 6

Share this post


Link to post
Tuna
3 hours ago, Lemmen said:

What did we do to deserve you? <3

Cannot agree more.

@DevTeam   Thank you for taking the time out to fill us in on what is going on, it's really great that you can be so open with us, and it's very reassuring to know that our issues are being looked into!

  • Like 2

Share this post


Link to post
MrWong

Thanks for implementing my lightbar suggestion, the units themselves look really sleek. However, are the blue lights going to be moved up to the bar at any point opposed to the front grill?

  • Like 2

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×