Comparing Shannon entropies and standard deviations of the order statistics for uncensored, semicensored, and censored distributions

Hayat Kılıç, Atıf Evren

As well as being three maximum entropy distributions, uniform, exponential and normal distributions have different properties in terms of censorization. Uniform distribution is censored from two sides . Exponential distribution is censored from below, whereas the normal distribution is uncensored. In this study, Shannon entropies of order sta-tistics  from uniform, exponential and normal distributions are considered.It has been found that  entropies of order statistics are some functions of the entropies of parent distributions.They are also functions of sample size, and  order of the statistic. Entropy estimates are then compared with standard deviations of order statistics. It has been detected that  for order statistics of censored distributions, like uniform or exponential distribution; as the two measures of  uncertainty;  standard deviation, and  entropy  do not convey the same information,  i.e., they are not positively correlated.This is  probably because the lowest and/or highest order statistics do not show high variability due to censorization.  For uncensored distributions like the normal distribution,  entropy and standard deviation are positively correlated as expected a priori.  Therefore in case of censorization using entropy statistics may not be appropriate to measure uncertainty or variability of order statistics.   

https://dx.doi.org/10.31873/IJEAS.5.10.25