We don’t know yet how the Vinsmokes will end up in the Whole Cake Island Story. Lot of the readers/watchers want Judge to be death, simply being punished for his actions against Sanji in the past and because he tried to force him for a marriage.
It all started already. The first steps were that he got k.o.’ed by Big Mom, and right before he was crying that he got betrayed like this (if the Strawhats weren’t there, he would have bitten the grass already with his kids).
Then Sanji threw some words at his head in Capone’s castle, that he should finally let him and that he is not his father in his eyes. Judge accepted it and went out to fight with his kids.
Yet for a lot it’s still not enough punishment that he received. But there is the point that he had to realize how strong Sanji became (they will probably see even more Sanji skills when they are about to continue to take actions against the Big Mom Pirates soon).
Could be Sanji has to ”save” them again, make them finally completly realize what Monster he became in the case of strenght and skills. Judge has to see with his own eyes that his own ”son” Sanji who he abandoned and felt ashamed in the past and told him he should never ever tell to someone that his father is Judge, surpassed all of the Vinsmokes. His other sons can’t have correct emotions, while Sanji still can (Judge had to feel this too when they took it easy that they were about to get massacred by Big Mom Pirates). At the same time he will make his thoughts how his other kids would have turned out in strenght if he wouldn’t have modifized them. Say if they could have became way stronger than they are now along with Sanji, then he would have such mighty army in his hand. But he simply failed and ruined it all at his own. This will make him living in the biggest regret for the rest of his life (even if he won’t admit it, but he will have to wake up with this feeling each day from now), knowing if he just had the patience to raise Sanji and kept training him in a normal way, even if he was way weaker than his brothers/sister, that he would have such monstrous weapon in his Germa Army by now. But it’s all too late, Sanji goes his own way with his crew, his true family who will always accept him. Ironically Judge always wanted that Sanji goes his own way, as he thought he won’t become strong enough to survive. That will be his great punishment that he has to live with it until he will poorly die.
*Theory by Qwattro