 What sort of white rock with lots of small holes is this?
 How to represent dimension$(0,n)$ matrices
 Issue plotting wormhole
 How to recover from this kind of startup error without reinstalling Mathematica?
 Odds on winning both hands running it twice when massive dog
 E. Coli metabolization of paracetamol
 Immune system vs pathogens
 What mutations in the DNA replication machinery would decrease the length of Okazaki fragments?
 Does the cell have a mechanism to determine DNA sequence from protein?
 How to search NCBI in bulk for a list of accession numbers?
 What is the grammar behind "上去“ in 听上去不错?
 What happened to the other dog?
 Relating two arrays
 Conservation law and hyperbolic conservation law
 What are the dimensions of a technic axle?
 Was Abu Simbel also designed to serve as a border between Egypt and Nubia?
 Did the Greek Theater in Taormina, Sicily also serve a strategic purpose?
 Is there an generally positive / appraisal / emotion boundary?
 Difference between forms of the georgian verbs with and/or without objective version vowel
 Bitcoin has vanished for 3 weeks?
Covariate shift detection
Is there any standard approach for detecting the covariate shift between the training and test data ? This would be useful to validate the assumption that covariate shift exists in my database which contains a few hundred images.
There are methods like the KullbackLeibler divergence model, the WaldWolfowitz test for detecting nonrandomness and covariance shift.
A simple test for quick analysis of covariance test would be to build a machine learning model, where the model is repeatedly tested with inputting training data and the production data.
In case, the model can make out the difference between the training and production datasets then it can be a sign of covariance shift.
Adaptive learning with covariate shiftdetection for motor imagerybased brain–computer interface
http://link.springer.com/article/10.1007/s0050001519375
EWMA model based shiftdetection methods for detecting covariate shifts in nonstationary environments (http://www.scienced

There are methods like the KullbackLeibler divergence model, the WaldWolfowitz test for detecting nonrandomness and covariance shift.
A simple test for quick analysis of covariance test would be to build a machine learning model, where the model is repeatedly tested with inputting training data and the production data.
In case, the model can make out the difference between the training and production datasets then it can be a sign of covariance shift.
20170717 11:51:58 
Adaptive learning with covariate shiftdetection for motor imagerybased brain–computer interface
http://link.springer.com/article/10.1007/s0050001519375
EWMA model based shiftdetection methods for detecting covariate shifts in nonstationary environments (http://www.sciencedirect.com/science/article/pii/S0031320314002878)
20170717 11:58:54 
Here is a simple procedure you can use:
learn a classifier to distinguish between train/test data (using regular X features)
compute the phi correlation coefficient to estimate the quality of the classifier = the separability of the train/test data
set a threshold (e.g. .2) above which you can claim there is a covariate shift (and start looking as corrections)
20170717 12:30:57 
You don't give many clues about what properties of the images you might be considering, but it seems that what you might want to measure is the difference in the distributions of the training and tests sets. A useful place to start would be with the Kullback–Leibler divergence, which is a measure of the difference of two distributions.
20170717 12:46:02 
The problem of covariate shift ultimately results in datasets with different underlying mathematical structure. Now, Manifold Learning estimates a low dimensional representation of highdimensional data thereby revealing the underlying structure. Often Manifold Learning techniques are not projections  therefore, different and more powerful, than standard PCA.
I've used Manifold Learning techniques (for eg: IsoMap, MDS, etc) to visualize (and, if possible, quantify) the "(dis)similarity" between train and test datasets.
20170717 13:03:53