"Paying Attention Where Attention Belongs"