Wash and dry potatoes. Slice potatoes thinly using a mandoline slicer like this one.
Place potatoes in a pot and add water to cover by 1 inch. Bring to a boil. Add salt and turn to simmer until potatoes are tender about 6 minutes or so.
In the meantime, add vinaigrette ingredients to a small bowl and whisk until well-combined
When potatoes are ready, remove from heat and drain. Place potatoes on a tray or baking sheet and immediately add the vinaigrette (while potatoes are warm), toss to coat. Let potatoes sit for 10 minutes until flavors meld.
Add onions, fresh herbs, and capers. Toss gently to combine.
Transfer potatoes to a serving dish or bowl. Serve at room temperature.
Make-ahead instructions You can make this mustard potato salad ahead of time. Dress the potatoes with the vinaigrette but hold the onions, herbs and capers until later. Refrigerate up to 8 hours. Before serving, add the onions, herbs and capers and toss. Bring to room temperature before serving.