Question: Is it the responsibility of the pastors or the people?
Answer: Yes! It is Both.
Someone very close to me privately addressed his church's "leadership" regarding matters of questionable doctrine.
He was scolded by one of his pastors, who told him that "it is NOT the responsibility of members to be concerned about doctrinal error."
Really? Paul commended the Bereans for “searching the Scriptures daily” to determine the truth of doctrine. Those were members Paul was addressing. Woe to those who look to their leaders for truth without checking what they are taught against the Scriptures.
My friend's pastor also told him that the pastors determine God's will for their church by "listening for a still small voice” and the member were responsible to follow for the sake of unity.
One can only hope and pray that such is not true. Of course pastors need to be listening to the Word of God, but NOT a “still small voice,” which is not unlike the “burning in the chest” that Mormons claim to have.
It is clear from scripture that pastors have the direct charge and larger responsibility to teach true doctrine, identify and warn about false doctrine, and protect their "flock from wolves." But the people also must be on guard and vigilant. Sometimes the wolves are on their pastoral staff.
When it became clear that my friend's concerns were met with resistance, he made the right decision to leave his church.