After having received a demerit point or demerit points, it takes a while until you will have your dice back. And you have to stay demerit free to make this happen.