Determining Communications Effectiveness

[Former Member]

We all know how critical effective communication to all stakeholder groups is to properly setting project expectations and keeping momentum going. Key activities include putting tougher the communications plan and then executing on it. However, one of the things I find most difficult is determining the "effectiveness" of the communications. First, it is helpful to know if the communication was "read", which is not always easy to determine. In the case of e-mail, we have all opened one and then not read it. Second, was the message that you thought you sent successfully received and internalized by the end stakeholder?

One technique that I have used to solicit feedback is to have an extended OCM team within the business and project communities which provide feedback. Surveys are also helpful, if the culture of the company supports them. One client included a drawing for a prize for all individuals who answered the survey to encourage response to help determine the general perception and knowledge of the project.

Please share your most creative techniques for soliciting feedback on the effectiveness of communications.

[Former Member]

Hi Caryl

I was watching this interesting thread closely. The replies received from experts prove one point. No one is sure enough how to measure the effectiveness of communication. In other words how much has gone into one’s head? That’s not an easy job. There may be scientific tools available but I am not a PR expert. I therefore put my views from a layman’s perspective. Let us tackle the situation like this. Imagine what would be the impact if the targeted community fails to comply with your suggested method? What are the risks involved? Does this risk factor concerns the stake holder? If it is the risk of the stake holder then probably we are targeting the right people; and your job is half done. Measurement also becomes easier. As has already been mentioned by others, the age old tool to measure effectiveness are the ones like survey, exit poll, questionnaires etc. Measuring effectiveness has a direct relationship how involved are you with the end user community. Think of your children with whom you are communicating daily and you can see the direct result and judge the effectiveness because you reward, punish, value and correct his/ her actions. These simple techniques work in industry also. The head of your communication department and indeed the head of your organization must signal his/or her commitment to program evaluation by a willingness to act on outcomes. That means communication program must be evaluated continuously while allocating budget & resources. When people see the result of his feedback being put into action, they feel part of the whole chain / company. It also means using the results of program evaluations in undertaking staff performance appraisals and generally giving credit where credit is due.

No communication is effective without these fundamentals. You must first set a target what you want to achieve after spending so much through communication. The tangible benefits must be very clear. If it is not clear the measurement becomes equally difficult. Let me give an example. Company A has put an ERP system in place. You want to communicate to the end users the benefits of reports available in Business Warehouse. Your employees hardly looks at these reports so are your bosses. The reason may be simple; the company does not need such reports or are dependent on other reports. While communicating you must be very clear about the benefits of these reports. The target audiences must be the top bosses. Here measurement becomes simple – one criterion could be how often the bosses are logging into BW system. Compare the data and Communicate again. This means the measurement technique would vary from case to case basis but the basics remain the same. If you find it meaningful and have some value please rate it suitably.

Regards

Alok Sengupta

[Former Member]

Most of the popular ideas have already been shared by our SAP gurus.

The popular methods that i have come across for measuring the communication effectiveness is a post communication survey, and the probability of the success of survey itself , is increased if it is linked with rewards.

Most of the posts have talked about "voluntary" methods. But in some situations, where time is short and making sure that the communication reaches and is read by the targeted stakeholders, it may be important to use a Forced Communication technique

I have come across one instance where, within 2 months, the organisation wide communication was to be done. They used a madatory certification test (based on the communication) to be taken. Each stakeholder had 4 opportunity (3 failure allowed), a cut off pass % was mentioned. The end user could take test in parts (as and when he had time) and given 1month to complete. The effectiveness of this activity was largley a success . The stakeholder group consited of about 40,000 employees!!

Coming on to add on to Modes of communication, Electronic modes are good, fast and easy. but at the same time, they may get just one opportunity , or a 2second window of attention. Coupleing traditional ways of communication sometimes gives a multiplier effect to the existing electronic means of communication.

for eg. a hard copy of the newsletter can be read anytime anywhere, a tabletop calender with communication, is seen 'n' number of times every day, a paper mat in cafeteria is good, pocket calenders stay with people much longer. Its about serving the communication to the stakeholder coupled with an item/service which has good percieved/actual utility (for the stakeholder)

[Former Member]

Thank you all for your contributions to what I agree is a critical communications activity. Effective and timely feedback is critical to determining communications effectiveness and to redirecting efforts when messages are not received as "sent".

Any other suggestions on global communications would be appreciated. Dealing with multiple cultures and languages further complicates communications and obtaining feedback. Any techniques or suggestions?

[Former Member]

I use a weekly 60 second phone survey. It is based on a small sample of the target audience, and depending on the phase of the project will be either very broad (e.g. measure awareness of the project), to more specific (e.g. do you know the go-live date, do you feel you have the tools to be successful).

[Former Member]

What you have highlighted is indeed one of the most difficult and understudied classes of Effective Communication. While we have studies on how to make communication effective we have very few papers on measuring it.

Apart from the well thought out techniques already highlighted in the community:

One way is to modify what Advertisers do: Measure Brand Recall.

We call up a few key people and check their understanding and recall of key communication.

Each Communication (Email especially) has a link to the Project website. Any one who actually reads the mail will be expected to click on that link. The site has a counter which will record the increase in the number of visitors.

We have quizzes on the website and participants receive a gift on answering correctly.

We create avenues for communication recall and registration. Key communications are channeled through at least 3 medias - Mail, Website and Posters. An individual can be expected to come in contact with at least one and reiteration of the message for those who come in contact with 2 or more.

Regards,

Roshik

[Former Member]

A fairly standard approach within the consumer products industry is to utilize an Organizational Readiness Assessment. This assessment asks a series of questions intended to guage understanding of the scope and impact the project will have and whether any actions are being taken in preparation for this. The results of theses surveys indicates whether the business is mobilizing as required.

An additional tactic would include Management Engagement workshops where managers for groups that are impacted are gather to discuss the specifics of how this project will impact their day-to-day operations. Having this as two way dialogue has proven effective at driving engagement and getting people moving down the required path. The difficult part of this process is determining when to conduct them as you need to have developed sufficient detail into your process changes to communicate to them, but you also need to be earlly enough to allow them time to action anything that comes out of the meeting.

[former_member583013]

I have used a similar approach in the past ... but combining "business trivia" and business fact. Each month we would send out an email-enabled survey that combined questions that tested the staff's understanding of the change initiatives underway with their knowledge of their leadership team. We offered a range of prizes for the first 10 correct responses. This is especially useful for large projects where the sponsors and the executives are at some remove as it tends to "humanize" the project and its ambitions.

Through this we began tracking some very useful metrical data. We found that the engagement rate (participation in the survey) increased over time, the response time decreased (how quickly the survey would be completed) and the data from the survey itself showed that our messaging was being internalized.

While this may not work for all organizations, it was a great success for us.

[Former Member]

Similar to what was previously mentioned, we have set up "communications liaisons" within the business community. We asked members of our target audience at the associate level (1 per location and/or stakeholder group) to give us 15 minutes a month. Every time a communication goes out, we call our comm. liaisons and ask what the "talk around the water cooler is."

Additionally, we had a place on our project website where questions could be posted anonymously. We chose 5 questions every Friday and had the project leadership team answer them for posting back on the website. We called it "Five on Friday."