Add flour, egg, milk, salt and pepper to the mashed potato and grated potato mixture. Mix into thick batter.
Melt butter in a non-stick pan or cast-iron griddle over medium heat. Scoop batter into pan. Fry until crispy and browned on both sides.
Repeat with remaining batter.
Enjoy with sour cream.