Early this year, I profiled a few associations that had recently restructured their chapter systems in the Associations Now Volunteer Leadership Issue. The International Association of Administrative Professionals, for instance, surveyed members and found that a poor member experience at the chapter level was a primary reason members were lapsing. Air Conditioning Contractors of America, meanwhile, had association chapters in about half of the states, and its membership was growing in states without chapters but declining in states with chapters.

In both cases, these associations knew it was time for change. They had evidence. They knew their chapters’ impact (or lack thereof) on their missions.

If your association has chapters or components, what evidence do you have? How do you know whether chapters are a lift or a drag on member value and the pursuit of your association’s mission?

Well, if you’re like most associations with chapters, you probably don’t know.

In August, Mariner Management & Marketing released the results of its “2016 Chapter Benchmarking Study” [PDF], a survey of 162 associations conducted in partnership with Whorton Marketing & Research. Among a trove of data about this less-studied side of associations was one perhaps troubling finding:

Just 5 percent of associations with chapters say they calculate the return on investment (ROI) of their chapter systems.

Meanwhile, the percentage of associations that measure chapter performance along individual metrics looks much better, but that’s only because 5 percent is so low. Still less than half do so:

• 41 percent said they measure chapters’ membership retention (this was the highest among the metrics asked about in the survey)
• 40 percent measure chapters’ member recruitment
• 27 percent measure chapters’ event participation levels

Peter Houstle, CEO/COO at Mariner, says this type of measurement is too often overlooked. “Lots of people have metrics by which they assess performance of the chapters, but those metrics invariably are around organizational metrics rather than mission metrics,” he says. “They’re focused on things like: Are you legal? Are your bylaws up to date? Do you have a certain number of meetings? Stuff that in the grand scheme of things represents activity but doesn’t necessarily represent value.”

These checklist items are easy to track, but more detailed membership and engagement data is difficult to share. If chapter and headquarters data systems are independent of each other and a volunteer chapter leader has to gather and send that info manually, that’s a process ripe for breakdowns.

“One of the fundamental problems that associations have when it comes to this value assessment and performance evaluation process is that, in the majority of the cases with associations with chapters, the data sets are fragmented. They don’t have a unified data set,” Houstle says.

About two-thirds of associations, however, process dues payments for their association chapters. Peggy Hoffman, CAE, president at Mariner, says taking on this role could make chapter-membership data collection easier.

“One of the low-hanging-fruit services is to be the dues collector, so you could actually get the data that would help you have a better understanding of how chapters are having an impact on retention and recruitment,” she says. “Another easy thing to do is to provide some sort of event-registration support, which now gives you really important information: You can take a look at who’s participating in events at a chapter level, which gives you a more compete picture, because someone could be ‘checkbook’ at the national level and very active at the local level, or vice versa.”

The challenge associations often face, however, is getting chapters’ buy-in to adopt recommended practices from headquarters. That’s why healthy relations between the two are crucial. “If there is a good symbiotic, collaborative relationship, then people would be looking at each other and saying ‘How can we do better?’” says Houstle.

A few other notable findings from the benchmarking report showed some of the common structural practices associations follow with their chapters:

Charter type. About two-thirds of associations’ chapters are formed so that the association has “substantial control”:

• 46 percent use a “separate chartered” model, in which chapters are separate nonprofit organizations but cannot exist independently of the central association.
• 20 percent use a subsidiary model, in which the chapters and central association are all one organization.
• 29 percent use a “separate affiliated” model, in which chapters are separate and could exist on their own if they chose to.
• 5 percent responded “other.”

Membership requirements. Here, associations are broadly mixed:

• 31 percent use a unified membership structure, in which membership in a chapter and the central organization are packaged together.
• 29 percent require membership in the central organization before a member can opt to join a chapter.
• 11 percent require membership in a chapter before a member can opt to join the central association.
• 20 percent are “non-contingent,” in which a member can join at either level with no requirement to join the other.
• 9 percent responded “other.”

Hoffman says she sees the unified structure growing in use among associations as they revamp their chapter structures, which she says is “an interesting counter to the fact that so many associations are looking at membership models and they’re asking the question, ‘Should we unbundle? Should we have more variety so people can have choice?’”

Staffing. Associations reported an average of 3.3 full-time employees at headquarters significantly devoted to chapter relations. And 65 percent said they have no full-time employees at their chapters, meaning they are dependent on volunteer leadership. “You realize how important the staff at the headquarters can be,” Hoffman says.

Overall, Houstle and Hoffman note in the benchmarking report an “undercurrent of discomfort with the status quo on chapters.” While the data in their report can offer associations an understanding of how their chapter-relations practices compare to other associations, a stronger commitment to performance assessment and measuring ROI of chapters will give an association a clearer picture of how they do or do not advance its mission, Houstle says.

“The problem is, if you ask this question, what do you do if you don’t like the answer?” he says. “I suspect there is some inherent fear of the answer, because some organizations have invested a lot of energy into [chapters]. It’s the classic sunk cost problem; they just can’t get around the notion that these things could be not worth the effort.”

Does your association measure the ROI of its chapter system? If not, what metrics do you track? What are the challenges you face in measuring chapter engagement? And how do you compare with Mariner’s benchmarking findings? Share your thoughts in the comments.