There is a bug with the ingame item, the Ogre Pet Whistle, in which the summoned creatures do not despawn appropriately.
Thank you for the reports! This has proven to be quite the complicated bug (it’s actually multiple issues on our side); so we don’t have an estimate on when it will be fixed yet.
I’m afraid you’ll have to keep living with the animals for a while. :P