Commonly referred to as ag. centers, agricultural extension buildings, or simply extension offices, these are a fixture in rural areas of the USA. They are the buildings that house the local agricultural extension programs, a collection of services and resources that support local farmers, gardeners, and related interests. They are generally built and funded by a combination of local government funds, the United States Department of Agriculture (USDA), and, often, local universities.
While local needs and offerings can vary greatly, you are likely to find organizations like the Master Gardeners, the Boy scouts, and 4-H clubs running events; a test garden or arboretum (sometimes more an honorary title rather than a true arboretum); and a chunk of public space that gets used for whatever local government agencies need. Sometimes space is available to rent, so things like gun shows and local fairs may be held there.
While less apparent to most of us, the ag. center will also provide information, training, and resources to local farmers, acting as an ongoing source of continuing education and a local clearing house of relevant information.