With over applications to the awards for , the exceptional strength and breath of moving-image practice in the UK reiterates the importance of this support.
We are delighted to be working with Guy and Reman over the coming months to realise their thought-provoking proposals and look forward to presenting the final works in the gallery at Jerwood Space next year. He has recently been shortlisted for the Jarman Award Hough is now in the early stages of production with a new commission with FVU, The Black River of Herself, which will be released in Looking for our exhibition programme?
You can find out more about our exhibitions and events here:. News Home. In layman terms, it is like Marty McFly from Back to the Future traveling to the future, getting his hands on the Sports Almanac, and using it to bet on the games of the present. Since time travel is still a few years away, hindsight bias is a serious problem today. Machine Learning algorithms take center stage today in artificial intelligence applications. There is a race to gain a fraction of a percentage improvement in model accuracy by optimizing modeling algorithms or inventing new ones.
While that is useful, you can achieve a bigger bang for your buck by focusing where the bottleneck is for applied machine learning, specifically with enterprise data. Hindsight bias is one such area mostly unexplored. So, how can we address this problem?
There is a suite of statistical tests that we can run on the input features to detect strong association of the features to the target label. Pearson Correlation provides a numeric measure in the range -1,1 between the feature and the label, which expresses how strong the association is between the feature and the label as well as the direction. While it works great for numeric features, it can work for categorical features as well once they are vectorized. However, if categoricals have a large number of unique values e.
This can be addressed by CramersV , and hence it is a more preferred statistical test for categorical features. The impact of such biased features might be trickier when it affects a small fraction of the examples.
Imagine a global geographic data. Lift is an alternative measure which catches such sparse hindsight bias. One strategy which has turned out to be useful is to perform some preliminary feature engineering before running statistical tests on the input features. For instance, many categorical features with hindsight bias follow the pattern of being null, until the target label is determined. They tend to have some value filled in, close to the when the label is specified. The boat and body fields from the Titanic data are examples of this pattern.
The way to bust them is to add a null indicator isNull derived feature and use CramersV as a statistical test.
Correlation does not always catch numeric features with hindsight bias. For example, in the context of predicting whether a sales opportunity will be won or lost, there was a feature called expected revenue. The system filled in the value after the salesperson closed the opportunity.
When the salesperson lost the opportunity, the system calculated expected revenue as 0 or 1. Otherwise, the system calculated it as a large number. A decision tree can be used to discover the two bins: [0,1] and [2, infinity]. Once you bin a numeric feature, you can treat it as a categorical feature. A statistical test like CramersV can then reveal the strong association between the specific bin and the label, thus exposing the bias. The other noteworthy pattern that we observed: categorical features disguised as text. For instance, while predicting whether a deal will be lost or won, there was a feature called Lost in stage.
Clearly, heavily biased, it was defined as a text feature but with only three possible values.
Editorial Reviews. About the Author. Being cheated on by a spouse is the second worst thing to Hindsight (Realworld) - Kindle edition by Doc Realworld. Hindsight: How to Catch a Cheater. [Doc Realworld, W. R. Logan] on Amazon. com. *FREE* shipping on qualifying offers. How do you catch a cheater?.
A cardinality check on such features, converting them to categoricals and then trying CramersV like statistical tests can reveal hindsight bias. Sometimes, the most elusive hindsight bias may not get exposed to the techniques above by just looking at the training data. One of the principal assumptions behind training a machine learning algorithm is that the data used for training is similar to the data used for scoring. Since features with hindsight bias contain information about the label at the time or right before the actual label is determined, we can look at the distribution of features in training data and the scoring data before knowing the actual label.
If any of the features surface a statistically significant gap in the two distributions, that is a candidate for hindsight bias. Temporal or timestamp cutoff is a related technique. Here, we determine a cutoff timestamp as the moment in time when the prediction event is supposed to occur, based on current and past records.
We then exclude all data before the event of interest, so we are not using any data that we collected close to the prediction or after, i. It is crucial to perform all data preparation and feature engineering within each cross-validation fold. As an example, if we use the label info in any feature engineering step, such as binning, we inherently introduce hindsight bias in the data. The same applies for feature selection, outlier removal, encoding and feature scaling methods for dimensionality reduction.
If we perform any of these on the entire data before cross-validation, then the test data in each fold of the cross-validation procedure played a role in choosing the features, and this introduces hindsight bias in the data. Across all the methods discussed so far, the hardest aspect is discovering the right threshold for your data and use case, which would help you reveal hindsight bias.
What should be the correlation measure, beyond which you mark a feature as biased? At what point do you say a feature is biased versus actually being a true predictor? You need to make the same decision on all other statistical measures, including the difference in the training and scoring distribution and so forth.
At Salesforce Einstein, our experience in building models for a wide variety of use cases and data of different shapes and sizes helps inform acceptable thresholds. Or should regulatory agencies restrict their actions to more lenient directives that incur minor calculated risks to public health—while keeping corporate profits up and consumer prices down?
When policy decisions can protect public health with little or no economic impact, barriers to making such decisions are minimal.
But when the first cases of B. Did this peculiar new affliction of cows present a real threat to humans? Did it even present a serious threat to cattle herds? No one yet knew for certain what caused B. In February, , a M. The committee supposed although it did not know for certain that the infectious agent responsible for B.
It knew that scrapie had been present in British sheep for at least years without ever crossing over into humans—or any other animal species, for that matter. Based on its interpretation of data available at that time, the commission issued an opinion that it would be "most unlikely that B.
That judgment—along with modest estimates of the number of British cattle likely to be infected over time—proved to be tragically, disastrously wrong. With acute hindsight, recent critics have been quick to condemn M. See, for example, the excellent New Yorker article cited in Resources.
But, for better and for worse, conservative judgments of the sort made by M. By curious coincidence, the mid 's was also the time when fierce battles were raging in the United States over public health recommendations related to another mysterious new disease. AIDS had appeared and had clearly begun to spread. But its causative agent had not yet been positively identified, and its modes of transmission were still under dispute. Frontline researchers had already convinced themselves that AIDS could be carried in blood and blood products; they were fighting an uphill battle largely behind closed doors to convince operators of major blood banks that new, more stringent and more expensive precautions were needed to protect the blood supply.
There again, a series of conservative decisions proved tragically wrong. Procedures to screen donated and purchased blood were not put in place until it was too late—and thousands of units of HIV-contaminated blood and clotting factor had infected unwitting patients and hemophiliacs.