Your surroundings may not be clean or there may be some nesting ground in the backyard. Even if a gutter or sewage if left open gives fly an opportunity to breed. And if your house is the closest, they may flock into yours. Use mesh door with springs. Also be careful if the flies have stings.