Forums » Bugs

File I/O failures causing unpredictable results

Dec 03, 2018 PaKettle link
I have noticed that heavy file i/o both read and write results in data becoming corrupted. It appears that attempting to execute multiple operations results in aborted transfers.

Some of this issue is plugin caused. I have been attempting to minimise this issue but it is impossible to coordinate file i/o between plugins and the main client. Unfortunately there is no mechanism to receive feedback on file i/o.

Since this issue appears to impact client data as well I suspect the best fix would be for some type of buffering at the client level. Alternately a call back or return value could be implemented so plugs could determine if they were successfull or not.

The issue does appear to be worse on some clients possibly due to the slower data storage phones and vr may have ( but I havent seen all the specs so thats only a guess on my part ).
Dec 05, 2018 incarnate link
Huh. Interesting. Do you have any example cases that demonstrate the issue?
Dec 07, 2018 PaKettle link
Hum... not sure what your looking for so pardon me being a bit off...

Client side issues appear to be losing my ignore list (and perhaps loss of my bind configuration)
I have also noted log files corrupted as well. It is hard to be specific as the issue is subject to the timing of the i/o activity and tends to strike in a semi random fashion and detection is always after the fact leaving the evidence overwritten. Fortunately client saves are mostly before and after most plugin activity lessening the chance of overlapping file activity.
(meaning client stuff gets hit less often)

On the plug side I am seeing partial save files and previously existing files completely wiped out.
I am know my current work which collects and saves a bunch of statistical data has greatly inflamed the issue for me currently which leads me to believe such issues may have been causing problems for some time. Previous versions only saved the data of a single sector at a time which usually only produced a moderate amount of data being transfered.
The statistical data however creates and saves a rather large table which is periodicly saved. (I am currently attempting to reduce the data to a more reasonable amount.)

I realise most of the issue is something I need to handle but I thought you may want to keep a eye out as it might also be causing some of the other random problems as well. Even with no plugs loaded you still do a bit of i/o and it still may be possible for multiple i/o transfers to interfer with each other.

I would send you my plug but I suspect a custom program that reads and writes to storage in a more controlled fashion would better serve you if you intend on stress testing.

Hopefully something in this mess will be of use.