From the BBC - The Tyranny of Algorithms
We know it well ....
Algorithms can be constructed to yield nearly any result the developer wishes. I use at least 200 algorithms to populate a financial projection. The result I want is a reliable answer. Algorithms, though, are used by people who don't even know they're using them.
The U.S. Department of Justice's Project Chokepoint is a blatant example. Algorithms were used to detect apparent fraud in banking activities. So far so good. The results were screened for industries with the most evident fraud. Maybe. Since now there was a list which could be targeted, the DoJ decided to folllow the money, and encourage financial institutions to deny banking services and loans to these industries. Whoops.
The list was then blatantly expanded to include industries that didn't align with the administration's ideology. That's beyond whoops. Industries targeted included telemarketing, pharmaceutical sales, coin dealers, pornography (first Amendment anyone?), ammunition sales, firearms sales, fireworks, mailing lists, pawn shops, payday loans, dating services, tobacco sales, travel clubs, surveillance equipment and several others.
Federal Financial Services regulators teamed with the DoJ to strong-arm banks into refusing services and loans to individual firms against which there was no proof of anything illegal, or even unethical. This denial of due process was heavily criticized by industries and individuals, but the government hid behind the mantra of "Protecting the Consumer."