63 lines
6.8 KiB
TeX
63 lines
6.8 KiB
TeX
\chapter{Conclusion}
|
|
|
|
%repeat motivation
|
|
StackExchange introduced the the \emph{new contributor} indicator as one of many efforts to improve the quality of the interactions in their communities. The indicator is shown in the answer box if the question is from a new contributor. The indicator reminds the answerer that the question was stated by new user who might not now the rules that well, and also to mind the code of conduct when giving their answer. This measure is supposed to increase the friendliness and welcomingness towards new users, while also be more forgiving when new users break a rule or convention unknowingly.
|
|
|
|
The motivation of this thesis is find out if the \emph{new contributor} indicator actually achieved its intented goal of getting the communities to be more welcoming towards new users and giving new contributors a good interaction experience. To measure this improvement, this thesis uses 3 properties: the sentiment of the answers, the vote score of question, and the number of questions from new contributors. To measure the sentiment, the answer texts are passed to VADER \cite{hutto2014vader}. An Interrupted Time Series analysis is then performed on the all 3 properties.
|
|
% chage is 1 of many efforts to make community better
|
|
% describe change shortly
|
|
% goal was to find out whether change improves user interaction experience for new users
|
|
% 3 criteria, sentiment, vote score, number of questions
|
|
% eval: vader -> sentiment, ITs, by looking at the 10 largest communities (list here, note so is by far the largest)
|
|
|
|
%repeat datensätze
|
|
As StackExchange has over 150 communities, this thesis only focuses on the 10 largest communities: StackOverflow, AskUbuntu, ServerFault, MathOverflow, SuperUser, stats.stackexchange.com, tex.stackexchange.com, unix.stackexchange.com, math.stackexchange.com, and electronics.stackexchange.com. StackOverflow is the largest community by far, more than 10 times the size over 2nd largest community math.stackexchange.com. The other communities are closer together in terms of size.
|
|
% 10 largest se communities
|
|
% stack overflow by far the largest, >10x times math.se
|
|
%
|
|
|
|
%repeat results
|
|
% change seems to be a success, not a one size fits all solution that works everywhere but a majority of communities benefit
|
|
% benefitters
|
|
The change introduced by the StackExchange team is a success, although it does not work for all communities. The new contributor indicator is not a silver bullet that is the single solution that works on all communities. The change produced desired results in more than half of the investigated communities: StackOverflow, AskUbuntu, ServerFault, stats.stackexchange.com, tex.stackexchange.com, and unix.stackexchange.com. In general, the number of 1st questions for new contributors increases in all of these 6 communities, while the number of followup questions increase in most of these communities. The sentiment values of the answers rises in all of these communities except for ServerFault. The vote score analysis does not yield meaninful results, either the vote score is not affected at all or changes drastically before or after change. Only for ServerFault the time matches and there a huge spike, but the vote score quickly returns to previous levels.
|
|
%in general
|
|
% first questions improved in all, followup in too in most of these communities
|
|
% sentiment raises in all except ServerFault
|
|
% vote score is does not yiel meaningful results, either completely unaffected or votescore changes drastically before or after change, only for ServerFault the time matches and there a huge spike, but it quickly returns to previous levels
|
|
%
|
|
|
|
% no benefits/no evidence
|
|
MathOverflow, SuperUser, math.stackexchange.com, and electronics.stackexchange.com do not profit as much from the change and show not an increase but decrease or continuation in the decrease of sentiment. The picture is not clear for these communities, only one statistic improves, compared to the the rise in the benefitting communities where the measured properties rise accross the board. The number of 1st questions decreases for all for these 4 communities and the number of follow-up questions also decreases for all these communities except for math.stackexchange.com. Similarly to the benefitting communties, the vote score is either not particularly affected or it changes drastically before or after change. The sentiment does not improve in these communities.
|
|
% the picture is often not clear for these communities, only one statistic improves, in contrast to the benfitting communtites where stats rise accross the board
|
|
% 1st question decrease for all communities -> bad sign
|
|
% follow up questions decrease for all except math.se
|
|
% vote score on a downward trend for all (for some even worse after)
|
|
|
|
|
|
|
|
%repeat discussion
|
|
%
|
|
Some of the investigated communities have intresting features in their data. In half of the communities seasonsality \cite{bernal2017interrupted} can be detected visually. In most of these communties the month of December signifies a month with less communty interactions wich may be attributed to the holiday season at the end of December. SuperUser saw a huge influx of new contributors shortly after the change who asked a lot of questions and dropping the sentiment and vote score value during that period. Stats.stackexchange.com has a similar pattern, although not as pronounced. This effect is also described in \cite{lin2017better}.
|
|
% one day files, state communities
|
|
% seasonality, state communities, state months
|
|
% large influx, state communities, state gravity
|
|
|
|
%other odd things (peculiarities)
|
|
The results of the StackOverflow community most closely resembles the expectation of improving the welcomingness and also most closly matches the example Interrupted Time Series shown in section 3. StackOverflow is also the community with the most dense and stable dataset while MathOverflow is the community with the sparsest data set. The sentment levels vary greatly between communities, stats.stackexchange.com has highest, while tex.stackexchange.com has lowest. The vote score level is the highest on MathOverflow by far.
|
|
% - stack overflow most stable, expected as largest community, also closly matches section 3 example
|
|
% - MathOverflow sparsest
|
|
% - sentiment levels vary drastically between communities, stats has highest, tex lowest
|
|
% - same goes for vote score, MathOverflow highest by far
|
|
|
|
%closing sentences
|
|
Overall, the new constributor indicator is a success and the majority of the communties benefitted from it. It is not a silver bullet solution that works on all communities. There is no simple solution as the communities are too diverse and the new constributor indicator is only one of many measure StackExchange has taken to improve user experience in their communities.
|
|
% change worked over all, majority benefitted
|
|
% no silver bullet, no simple solution, no 1 size fits all, communities to diverse
|
|
|
|
|
|
|
|
%sum up findings
|
|
% change did something?
|
|
%...
|
|
|