Customer tests breaking the build?

I was chatting with a team member about the possibility of his team moving toward ATDD (Acceptance-Test-Driven Development) in which they would write automated acceptance tests before development. He raised the concern that it would mean that they would be checking in tests that broke the build for unacceptably-long periods of time (a day or more). I turned to Brian Button and Ryan Tyer to see how they handle that. Their responses:

Brian: I ignore the ATs that aren't implemented yet so they don't show up as broken tests. (I mean ignore from a JUnit ignore point of view, not just ignoring by choosing not to look at their failures!) Once I start working on something, I unignore the tests.

The restriction on checking in code on broken tests goes for programmer tests, not customer tests. Customer tests go from ignored, to failing, to passing, and shouldn't ever regress.

Ryan: I'd add that with some testing frameworks (TestNG comes to mind) you can flag tests to ignore programatically (@Ignore if I remember correctly). These will not be run so they will not fail the build. You could have two suites of tests (all and all but ignored tests) for the frameworks that don't support this.

Brian: That idea is what I was talking about. In junit 3, you'd have to comment the test out or something, since it doesn't have a way to ignore tests. In Junit4, Nunit, Xunit.net, TestNg, you just mark the test as ignored.

In visual studio architect edition, team suite, or tester edition, you can use test lists to separate ignored ATs from those that should pass.