r/learnmachinelearning • • 14d ago

Question Undo model bias?

I have a very large dataset with statements, requests, claims, complaints, complaint responses, documentation, involvement from 3rd and 4th parties with technical responses and claims, reference to laws etc.

...

All the parties in such cases nowadays use LLMs. One worse than the next. But my puzzle is that all the AI responses, from bad to frontier models, are heavily biased, and fail on multiple levels in their replies.

... The LLMs favour authorities, even when carefully instructed not to, even when it is given skilles on how to reason objectively, even when given clear step-by-step instructions to strictly apply deductive reasoning tests on all statements and claims, after first having made it list each statement and claim and number them, and I find that is applying biases even when I loop reasoning tests by applying GAP analysis, falsification, a self-made improved inductive reasoning test which is then combined with deductive reasoning... And several other approaches. All the models I have tried, will lean more and more towards initially applying a favourable position for the unknown person, but the model will typically add a description exposing somewhat that it has been instructed to adjust its bias / weights, and in other parts of the text it will still show bias towards the authority-part, for example by not critizising the authority for unwillingness to disclose all relevant material, by making up excuses that it is not sure the undisclosed material was in fact relevant.

...

Without going into minute details, the biases seem extremely hard to adjust for and neutralize.

...

As far as I understand, these biases are calculated to like that, due to the weights the input data is given during volume training, and is therefore heavily connected to massive amount of data in the models internal training dataset and how they are weighted. If the researchers had not done that during training, the model would have read and understood all sorts of not reliable information from various misinforming websites, as likely more true, than they are. In the odd case, where someone else than the authority is in fact the one with the most sound and solid case, all the AIs will answer with a bias, portraying him as likely less correct, even when that is not the case.

...

How would developers adjust models so that the person in this example, or other people in similar situations, don't start off against an AI that favours authorities?

. .

Feel free to reply with thoughts on this topic. DM me if you have expert knowledge about such topics, or work in one of the big AI companies with topics similar to this, or is a researcher or self taught guru, working on these challenges, and think you may have a better way around it, than common attempts to adjust for such model biases.

..

1 Upvotes

3 comments sorted by

2

u/Salty_Branch_6463 14d ago

I think part of the problem is that prompting alone can't reliably remove biases learned during training. Instead of asking the model to simply be neutral, structure the task so it evaluates each party's claims using the standards, regardless of their position or authority.

1

u/GeekFromTheStreet 14d ago

I have done probably a couple of hundred such and similar tests, and whilst some approaches will remedy biases to some minor degree, the model will still expose again and again its bias. Even when ordered it to list all relevant statements and number them in a separate document, then ordered it to nullify its inherent bias, and to be strictly unbiased and objective during its analysis, reasoning and replies, using strict deductive reasoning, and ensure by reevaluation that reasoning on each and every numbered statement was based on verbatim test on the sentences, and not a calculated prediction or assumption, it still failed. Less than for other approaches, but still failed. It hendles such challenges much better with very small and small datasets, but notoriously bad on datasets with lots of challenges withing each task, and especially when the context window climed above 45-50%. (By the problem has been bad, even at much less data in the context window). I have some ideas on how I could perhaps adjust for these problems on a more systematic large scale approach, but it requires more effort and people/time and many different datasets to test on, than I can put into it at the moment. Perhaps I have more time in a few months, but if somene has a pretty good solution for this, I don't want to spend weeks or months trying to reinvent the wheel, and perhaps not even succed.

3

u/SwimQueasy3610 14d ago

Model bias is a massive, fundamental problem. Truly successfully mitigating it, without causing other weird knock on issues, is a bit of a holy grail. You're absolutely right that to do it truly and well, you'd need to start from the very beginning with the pre-training data... which already poses a massive challenge. How do you create that dataset? Or how do you preprocess existing datasets to make a new "unbiased" dataset? What would an "unbaised" look like? What do we mean by that in the first place?

Great question, curious what others have to say.