The bargaining unit is a subset of employees who will be covered by a collective bargaining agreement.
It is a common misconception that employees at "union shops" all work under a union contract or are even all union members. This is not necessarily the case, and indeed, it is often quite the opposite.
Bargaining units are significant for two major reasons: first, once union representatives reach a proposed agreement with management, only members of the bargaining unit are allowed to vote on the measure. Second, once a new contract is in place, any protections or benefits afforded individuals within the bargaining unit are not inherited by people outside of the bargaining unit.
Okay, so why is this really so important?
Well, besides the obvious "not everyone benefits from the new contract" problem, employers often attempt to use the makeup of the bargaining unit to affect the outcome of a union vote. By affecting who can and cannot vote on a particular issue, employers may gain the upper hand in squelching support for an initiative unpopular with management.
For example, consider if workers are attempting to bring in a union for the first time. Union organizers rally support behind their cause and get the requisite number of union cards signed by employees, prompting a certification election by the National Labor Relations Board. At this point, the employer may opt to challenge various employees' candidacies as members of the bargaining unit, trying to get certain workers included and others excluded.
The employer does this in hopes of getting known union sympathizers excluded from the bargaining unit, while getting known pro-management types (or even members of management themselves) included in the bargaining unit. While the NLRB makes the final decision regarding who should and should not be included in the bargaining unit, employers can often greatly enhance their chances of avoiding unionization by putzing around with the makeup of the bargaining unit.
In the end, of course, it all comes down to a simple vote. If the employer (through NLRB challenges or unfair labor practices) is able to get enough anti-union employees into the bargaining unit, then the certification vote will fail and the union will not be allowed to represent the employees for purposes of collective bargaining.
Certainly, this doesn't always work in favor of the employer (especially if there is broad support for unionization amongst the employees, which there often is), but manipulating the bargaining unit--stacking the deck, if you will--is a common anti-union technique employed by almost all firms during labor negotiations.