FunctionalCommunities

https://Etherpad.Mozilla.org/measure

#FunctionalCommunities

    - ownership - everyone owns
    - measurable: maybe
    - helpful to each other
      - measurable: maybe
    - new ideas welcomed
      - measurable: maybe
    - can ask questions
     - questions get responses
     - it's ok to not know
        - measurable: probably
    - ok to be new
        - measurable: probably
    - rallying point
        - measurable: maybe
    - releases get shipped
        - measurable: yes
    - positive sentiment
        - measurable: maybe
    - contributions get thanks
        - measurable: probably
    - acknowledge - accepted or rejected, not just sitting there
        - measurable: yes
    - communication happens - know what's going on
        - measurable: maybe, via e.g. survey
            - similar measurements: Daily Show readers vs. other readers <http://www.people-press.org/2007/04/15/public-knowledge-of-current-affairs-little-changed-by-news-and-information-revolutions/>

- generates product - measurable: yes, though not automatically - welcome note to newbies - measurable: yes, by asking the community and (e.g. randomly) checking with newbies if they received it - communities - some people just “look” but don’t contribute - measurable: yes (though not sure what is the point) by measuring e.g. non-speakers on IRC or page views on mailing list pages - page views on mailing list pages do suggest relevance, which is good - lurker to participation - measurable: maybe - too much chatter - measurable: maybe, via e.g. survey - people do what they say - measurable: perhaps, by looking at when someone claims a ticket vs. when it gets closed, and by whom. Highly community dependent (?) - signal to noise ratio - criticisms on content not on person - measurable: perhaps yes, by which pronouns are used (?) (Asheesh thinks this is just awfully messy) - guidances on how to improve - measurable: yes, though maybe not automatically - power is distributed - measurable: qualitatively, through having someone look at e.g. an org chart - life happens - how to you step out and step back - community must be fun! Joy. - measurable: yes, via e.g. surveys. Be careful about randomness of sample. - people should be satisfied about conflict resolution - new contributors successfully joining the community - measurable: yes, by doing what Greg + Linux Foundation already does with State of the Kernel, looking at emails in git commit logs, and so on http://www.flickr.com/photos/7358986@N02/9109127966/in/set-72157634245175804 discourse discussion platform #DisfunctionalCommunities? - trolls - Decreasing relevance of the project - measurable: perhaps, via decreased user count, if you can measure user count - the hand of god management model - decisions just happen, they aren’t discussed - measurable: probably, by survey on satisfaction on decision process (again, pay attention to random sampling) - silencing people - measurable: probably not? - not valuing new contributors - measurable: to what extent do new contributors contribute >1 patch? - discounting minority members - measurable: perhaps, if you can identify posts by minorities by e.g. looking at names and see if they fare worse/better in terms of other metrics, like replies that contain the same content - put forward idea (does it get ignored or bounced) - have a man put forward the idea, does everyone say it’s a good idea now? - see above - anyone that is an outsider - is their voice valued - see above - measure dropped patches and dropped ideas - are outsiders ignored? - meaurable by dropped pull requests or dropped patches; then sub-filter by demographics, etc. Note that it is very easy for confounding factors to make this very difficult. - maintainer stealing code and not giving people credit for their contribution - measurable: maybe? look for patches committed that have similar text? - cost of contributing is hard - social cost to contributing - have to wade through verbose mails - have to pass tests - measurable: maybe? hmm - communication tools - divided communication - have email, IRC, mailing lists - but all decisions are made on IRC and they won’t post a log - can’t find where all the decisions are made - in person, twitter, etc. - accessible to everyone - measurable: maybe? hmm - people don’t feel comfortable participating - % of Unique Contrbiturs / Total Size of Community - Median participation points per person - too many people playing devil’s advocate - to the point that they break down the communication process - measurable: maybe? hmm - have a person that writes up what happens on mailing lists - what was discussed and what was decided - measurable: yes! - sentiment of email - measurable: yes, but see above concerns about sentiment analysis - group think - 20 people who have been in the community say “that’s right” but it’s not - measurable: no? - “protect the pack” - community members protecting long-standing members - measurable: no? - high turn over - in both membership and leadership - marked up tick after a troll leaves - measurable: yes! - no awareness of users - measure user participation on mailing lists (user defined as someone with no patches in the commit log) - will not answer questions or tickets, don’t have a survey - what does this mean? - subjective happiness scales - see research on ESM (experience sampling method: http://www.amazon.com/Experience-Sampling-Method-Measuring-ebook/dp/B004B8UXW2/) - be sure to sample very randomly - sentiment analysis - but need to verify that people are actually happy or sad - compare to their git commits - miss sarcasm - randomize time of when you measure - need to have a blackbox algorithm, or people will game the system - good heart’s law - when you make something a measure of project, it ceases to be a good measurement tool - profanity - measureable: yes - posts on the mailing list - long periods of silence - Data source: FLOSSMOLE has data dumps from this - five geek social fallacies - “we can’t not welcome trolls” - ?? - projects can get too big - dunbar’s number - ?? - talking about code vs people - use that as a measure - measureable: perhaps Tools - sentiment analysis - bluebird, already used by Donnie to measure Xorg + Gentoo - i.e.: https://github.com/redmonk/bluebird/blob/master/bluebird.py & https://github.com/redmonk/bluebird - metrics grimoire http://metricsgrimoire.github.io/ - do you have a code of conduct - published repercussions - https://openhatch.org/wiki/Project_codes_of_conduct - python code of conduct - open, polite, civil interrupters - correct behavior without giving positive or negative feedback - give them a penny - bot on maiing list: the penguin is here! - randomizes the From header - 5 or 10 of the top commiters to do this - orient around influence, not control - FWIW Asheesh is a HUGE FAN of this plan and wants to begin implementing it tonight or tomorrow. Sarah says YES! But let’s do this on Wednesday at OSCON “do this” == “find the people to sign up as people willing to say “penguin is here””? True Things needed: - List of 5 people signed up - bot notification system - suggestion: there is a semi secret email address that any of the 5 people can email – when you forward the message to that email address, it goes and finds the message in the archives and sets up reply-to etc. properly, and bottom-posts “The penguin is here” (other strings welcome) with a randomly chosen From. - I can write the bot momentarily if we want. - Anything else needed? - Also: I would like to replace “The penguin is here” with a different strong. Trying to come up with one. influence graph - who changed Linus’ mind - consensus: probably no one can change Linus’ mind - chain of conversation - who finally said yes to a patch? Who is the influence?