The term "bitter end" needs some explanation.
On a sailing ship, the bitts are the large timbers in the fore-part of a ship, that hold the anchor cables. (There are also after-bitts, for controlling the running rigging from the quarterdeck.)
If you bitt the cable, you are bringing it around the bitts. A bitter is one turn of the cable around the bitts. The bitter end is the end of the cable that remains fastened to the bitts when the anchors are out.
So, if you are "at the bitter end", your anchor cable is stretched to the utmost, probably dangerously. If you stick with something to the bitter end, you are faithful to one course of action without considering the consequences.