The row value(s) updated or deleted either do not make the row unique or they alter multiple rows

I was working with a Microsoft SQL database today (not mine thank goodness) that had duplicate rows; the table had no primary key defined. When I went to go delete the duplicate row (or even change it), I got the error:

The row value(s) updated or deleted either do not make the row unique or they alter multiple rows(2 rows)

It was somewhat hard to figure out how to delete the duplicate row. In the end, I was able to delete it by doing:

SET ROWCOUNT 1
DELETE FROM myTable WHERE statmentToSelectTheDuplicateRow

Another reason why to always define primary keys so you don't even get into this situation.