Place potatoes in a large pot and cover with cold water. Bring to a boil over high heat, then reduce heat to medium to maintain a gentle boil. Cook until fork-tender (but not falling apart), about 12 minutes.
Drain potatoes in a colander and allow to cool while you're preparing the other ingredients.
Stir together mayonnaise, mustard, 1 ½ teaspoons Old Bay seasoning, salt and pepper in a large bowl.
Transfer drained and cooled potatoes to the bowl with the mayonnaise mixture. Gently stir to coat potatoes. Add eggs, onion, celery and pickle cubes and gently stir again to incorporate.
Season with additional salt and pepper, if desired.
Cover and refrigerate at least 1 hour before serving.
Sprinkle with the remaining ¼ teaspoon Old Bay seasoning when serving (or paprika if not using Old Bay).