5

I've been looking around different sites's metas to see how they handle different challenges of figuring out how to maintain high quality questions and answers.

With some sites, like StackOverflow, it's easy to spot a good or bad answer. It's generally either right, or it's wrong. Questions either contain facts about the problem and evidence that the question asker has researched his/her question, or it doesn't.

Project management involves many soft skills that make it hard to tell what is right and what is wrong. The Workplace SE solves this problem by enforcing the "back it up" rule:

Please note that answers should be backed up either with a reference, or experiences that happened to you personally. You should always include in your answer information about why you think your answer is correct.

As an example of what this might look like, I tried this out myself on my two latest answers:

For hard skills, or for the more technical points of project management that are taken directly from the PMI or PMBOK, we could directly cite these with a link to the documentation.

Enforcing the back it up rule is something anyone in our community can do. Comment on answers that don't cite references and challenge them. Ask for references, evidence, or something to support the content included in the answer. If the answer cannot be backed up, use your downvotes as a tool to mark that answer as potentially incorrect. Upvote answers that do cite references and that do support the answer.

What do you think? Is this a viable solution for PMSE to ensure we're giving correct answers? Are experiences enough? Should we still cite references if explaining why we think our answer is correct? If not, what are the criteria, and how should we approach this Q&A problem?

Related: Draft of a new policy for our site

5
  • Well, we're trying to enforce "back it up" but so far we just have it in the FAQ and have it as an understood policy. I think the psychological effect it brings is helpful for those aware of it but "enforcement" hasn't been realized beyond the psychological impact.
    – Zelda
    Commented Jun 3, 2012 at 3:48
  • @BenBrocka - I think enforcing the "back it up" rule is something that can't be a moderator duty, it has to be a community duty. What I do here is if I see something that looks potentially bogus or questionable to me, I'll ask the poster to back it up. If there's no response in a reasonable timeframe, I may downvote.
    – jmort253 Mod
    Commented Jun 24, 2012 at 2:09
  • Agreed, but having it in the FAQ and agreeing on it on Meta is the most important part of it though. We don't really use any moderation tools RE this problem on Workplace aside from the citation needed post notice, but having it in the FAQ and the meta consensus is a strong tool. It creates a culture of cite your sources and gives users something to point to and say "you should cite your sources and this is why."
    – Zelda
    Commented Jun 24, 2012 at 18:59
  • @BenBrocka - If there is enough support from members here, I would definitely add that to the FAQ. I'm planning on addressing the issue of the FAQ shortly, once Anna posts her analysis of the self-eval.
    – jmort253 Mod
    Commented Jun 24, 2012 at 19:04
  • Might want to [Featured] this btw, I do that with any post I want attention from users outside the Meta regulars. It'll at least get it a bit more exposure.
    – Zelda
    Commented Jun 24, 2012 at 19:05

4 Answers 4

Reset to default
2

I broadly agree although I feel the idea of a correct answer is pretty flawed here.

I've got an agile background so will be far more inclined to answer a question with an agile slant, others with a Prince 2 background will tend to answer along those lines. Both answerers may be able to back up their answers with citations and experience, both may be equally right.

It's not uncommon to see questions with many different answers. The problem is that in order to zero in on one that is 'right' we usually need to know more about the context than we do or is even feasible to learn without turning it into a discussion. Equally, if we do question and learn more, the question becomes so much more specific that it's probably not useful to future visitors.

I feel it's best to encourage people to provide some context for their answer - this would add value to the multiple answers given both for the person asking the question to decide which might work best for them and also for repeat visitors who might have the same question in a very different set of circumstances.

Sorry if that seems like a semantic distinction but I think it's important.

2
  • Hi Ben, thank you for providing your insight. I see your point. The best answer I can come up with is that there's a balance between something that becomes too localized (not useful to future visitors) and not a real question (there's a lack of so much information that literally every single answer is correct). Judging these questions is subjective. I think you make a great point, and here's what I'm thinking: Questions don't have to have a single correct answer, but they should be reasonably scoped as to where not just anyone can post a non-answer.
    – jmort253 Mod
    Commented Jun 26, 2012 at 6:17
  • [cont'd]-I see nothing wrong with the examples you cite. Those are the questions we want. With that said, if you think I've made an error in judging a question, feel free to cite specific examples. Please post a meta question about it. Feedback is absolutely critical to the success of this beta site and moving it forward, as well as to help guide the community and get more people involved in its governance. I'm serious. If you see things we as a community can do better, please say something. Meta community participation on a StackExchange Q&A site is extremely important. :)
    – jmort253 Mod
    Commented Jun 26, 2012 at 6:28
1

Enforcing the back it up rule is something anyone in our community can do. Comment on answers that don't cite references and challenge them. Ask for references, evidence, or something to support the content included in the answer. If the answer cannot be backed up, use your downvotes as a tool to mark that answer as potentially incorrect. Upvote answers that do cite references and that do support the answer.

In addition to this being something anyone in our community can do, I want to point out that this currently isn't a moderator duty. It's not our job to determine if all answers are "correct". This is definitely one area where community support is a must.

Downvotes are perhaps the most powerful community moderation tool available to individual users of this site, and it takes only 125 reputation to utilize this feature. When combined with a polite, constructive comment, they can either change an undesired behavior for next time or help the poster correct the issue and formulate a very good, compelling answer.

This isn't to say that I won't personally downvote something I know is wrong or that I disagree with, but this is only something I can do in areas where I have the expertise as a member of this community.

1

I would be all for "back it up" guideline or even a policy, but not for its enforcement. Actually there is one crucial thing in what you write:

Comment on answers that don't cite references and challenge them.

What I believe we lack (and I'm first to admit I'm a sinner here as well) are healthy interactions regarding answers. I mean the only tool I use for answers that are half-baked is downvoting, but most of the time I just ignore them, hoping that they won't anywhere to the top and dissolve in oblivion.

This basically means I don't try to make them better, and with the simple guideline stated above most of them could be improved vastly if author wanted to invest some more time.

4
  • I don't think that the back it up rule is necessarily a moderation policy, it needs to be a community policy, which means the members of our community must challenge answers they think are wrong. We're fortunate to have a really mature group of people here; I think we can handle a little constructive criticism now and then, especially if it improves quality! :) Is this something you think you could commit to if other community members agree with this as well, for the benefit of PMSE?
    – jmort253 Mod
    Commented Jul 3, 2012 at 6:30
  • 1
    I think about it as of a community policy or guideline, and definitely something that should be done by the whole community and not only mods.
    – Pawel Brodzinski
    Commented Jul 3, 2012 at 6:41
  • Mark and I were just talking about updating the FAQ, so we can add this in. However, we're expecting this to be something that is enforced by the community. We can't rule out the strict guideline if the community doesn't step up. With that said, I much prefer community enforced guidelines as opposed to rigid moderated rules. I know we can make this work! ;)
    – jmort253 Mod
    Commented Jul 3, 2012 at 6:50
  • I just realized I did jump the gun on this a little bit. We may wait for other people to contribute this discussion before making a decision as there may very well be other points to consider. Thanks again! :)
    – jmort253 Mod
    Commented Jul 3, 2012 at 7:29
0

The biggest area for improvement for the site, according to the area51 metrics, seems to be the volume of questions, rather than the quality of answers.

2
  • The metrics actually just address the volume of activity. You're right that the metrics show continued movement forward, with the exception of questions, but the quality of both the questions and answers is something we will need to review. That's not to say the answers are bad or wrong, or that everything needs to be fact-based, but I think it's worth getting everyone involved and discussing the merits of the back it up concept.
    – jmort253 Mod
    Commented May 30, 2012 at 19:20
  • Mark, there is a great point in Ben's answer. If a question is reasonably scoped, the answers stand on their own. Maybe there isn't a single correct answer, there is still expertise. List questions "What software does X" fail in this area miserably, but the questions that Ben and DavidEspina describe are exactly the types of questions that should be embraced here. I'm happy to leave the "back up your answer" issue aside and just focus on question quality. (However, I still believe each user reserves the right to disagree/demand clarification :) )
    – jmort253 Mod
    Commented Jun 26, 2012 at 6:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .