Lost 100k Cash.

[Mycaelis]

Hi Guys,

After your latest update, I've just made about 100k today, sync'd my data and logged out.

Logged back in, and all the 100k I made is gone, normally I wouldn't squabble over 100k but now that's over an hours work that I've just lost.

In-game name: Mycaelis
 
What was lost: 100k in cash.
 
Value of item/money lost: 100k

Quick description of what happened: Did loads of work to make the 100k, Synced my data, even waited 10 minutes for the next indicator to come up with my data was synced, logged out, logged back in, and poof, all my money gone.

 
Let me know if you can help me get it back.

Cheers.

Mycaelis.

 

[Wilco]

Did you bank the 100k or was this in your hand (just so we can investigate the issue )

 

[Mycaelis]

I banked it all, my bank balance when I logged out was about 397k, when I logged back in, back to 295k.

I'm guessing the database just wasn't syncing properly as a mate of mine lost about 144k around the same time.

It was between 9:55 GMT and 10:55pm tonight.

Thanks for the assistance.

 

[Mycaelis]

Hi Wilco,

I think I got about 60k back when the server fully restarted.

Don't worry about the last 30 odd k.

Cheers,

Mycaelis

 

[Mycaelis]

Hi Again,

Just lost another 100k, definitely losing my will to play Altis Life...

- Made my money.

- Banked it.

- Sync'd immediately.

- Waited 35 minutes until server restart (was just flying about doing nothing much).

- Server restarts.

- Log back in.

- Missing the last 1 hour of work before the server restarted.

What is going on with these database rollbacks on restart?

Is it just not sending the data to the database? or is the database being rolled back over an hour or so after restart?

You really need to look into this whole database syncing issue as one of your top priorities...

I've probably lost over 1-2 million in the last few weeks just from the database rolling back constantly.

 

[TinyBigJacko]

It is being investigated as a top priority - along with everything else that is a top priority (anti-hacks, solving the weapons-drop-on-disconnect/death problem, economy-changes in response to popular outcry and TS server relocation - oh, and forum-checking and adminning the server).

I'm going to alter the way that Tonic's DB code is handled, so that we can start clearly timestamping each entered line into the DB. At the moment, we are dependent on separate logfiles from the game as it calls the SQL server, but there is no guarantee that these calls are being physically executed on the SQL server - just that the game 'has asked for it to be done.' I've had my suspicions about Tonic's code here for quite some time, but it is a horrible ball of spaghetti which I'm slowly unravelling.

Just to be clear, there is no 'rollback', per se, when the server restarts (whether the game-core has been updated or not). The server is SUPPOSED to just pick up and carry on, but clearly, sometimes, something is not being written to the DB. Trouble is, whenever we have stress-tested and analysed what traffic is reaching the database, everything comes back clean with no errors. It's possible it's a load/collision thing, in which case we'll have to stick some (potentially speed-knackering) SQL-server-side logging on as well in order to find out if it's a peak demand thing, but this is (necessarily) a big step, and has implications that worry us, if too much DB time is spent logging instead of writing data.

There is also a (slim, but nonetheless, existing) possibility that Tonic's code may be doing something peculiar either on re-connect/respawn, or it may even be down to whether you 'abort-disconnect' versus 'respawn' versus 'die', and whether the DB writes are handled differently in those cases. When I find out, I will fix it. At the moment, I'm still hunting, but at the moment, the mouse is beating the cat...

 

[Mycaelis]

It is being investigated as a top priority - along with everything else that is a top priority (anti-hacks, solving the weapons-drop-on-disconnect/death problem, economy-changes in response to popular outcry and TS server relocation - oh, and forum-checking and adminning the server).

I'm going to alter the way that Tonic's DB code is handled, so that we can start clearly timestamping each entered line into the DB. At the moment, we are dependent on separate logfiles from the game as it calls the SQL server, but there is no guarantee that these calls are being physically executed on the SQL server - just that the game 'has asked for it to be done.' I've had my suspicions about Tonic's code here for quite some time, but it is a horrible ball of spaghetti which I'm slowly unravelling.

Just to be clear, there is no 'rollback', per se, when the server restarts (whether the game-core has been updated or not). The server is SUPPOSED to just pick up and carry on, but clearly, sometimes, something is not being written to the DB. Trouble is, whenever we have stress-tested and analysed what traffic is reaching the database, everything comes back clean with no errors. It's possible it's a load/collision thing, in which case we'll have to stick some (potentially speed-knackering) SQL-server-side logging on as well in order to find out if it's a peak demand thing, but this is (necessarily) a big step, and has implications that worry us, if too much DB time is spent logging instead of writing data.

There is also a (slim, but nonetheless, existing) possibility that Tonic's code may be doing something peculiar either on re-connect/respawn, or it may even be down to whether you 'abort-disconnect' versus 'respawn' versus 'die', and whether the DB writes are handled differently in those cases. When I find out, I will fix it. At the moment, I'm still hunting, but at the moment, the mouse is beating the cat...

Thanks for the response, and in all honesty, I know how difficult these things can be to track down and resolve.

I certainly hope its just a bit of dodgy code as once its found, its probably a quick win.

Best of luck in sorting it, I think i'll take a break for a few days and let my Altis batteries recharge.

 

[Wilco]

130k back in your account, a fix is being rolled out for this so there hopefully wont be many loss of things going forward