If somebody upsets you regularly – or even occasionally for that matter – there’s something you can do about it without making a fuss or going public.
The english expression about goat-getting is used as in “he/she get’s my goat” or “that really got my goat”. It means that something annoys us, often in a wheedling, progressive way that builds over time.
If someone gets your goat the remedy is simple. In order to get your goat they need to know where it is kept, so hide your goat!
Do you want and explanation of the expression ‘Got my goat’?