Lydia is very controlling and often gives her children back-handed comments. Hilda and Zelda have a lot of issues with her that usually come to light in flashbacks and stories throughout the series.
In Mom vs. Magic, Hilda and Zelda spend a considerable amount of time debating over which of them is Lydia's favorite daughter. However, they find a note from Lydia saying she had gone out with Vesta, which leads them to believe that Vesta is in fact Lydia's favorite daughter.
It is revealed that when the kids were little, Vesta turned her mother and father into pigs because she was insanely jealous of the attention they gave to Hilda and Zelda. Because the Witches Council decreed that people with pig's feet couldn't properly raise children, Hilda and Zelda were adopted by a nice family on a farm. After Lydia head butted Vesta, she turned them back into people. And later, in order to legally retrieve Hilda and Zelda, Lydia adopted them.